Principles Of Statistics

• A1.12 B1.15

  Principles of Statistics

  Part II, 2001

  (i) What are the main approaches by which prior distributions are specified in Bayesian inference?

  Define the risk function of a decision rule $d$. Given a prior distribution, define what is meant by a Bayes decision rule and explain how this is obtained from the posterior distribution.

  (ii) Dashing late into King's Cross, I discover that Harry must have already boarded the Hogwarts Express. I must therefore make my own way onto platform nine and threequarters. Unusually, there are two guards on duty, and I will ask one of them for directions. It is safe to assume that one guard is a Wizard, who will certainly be able to direct me, and the other a Muggle, who will certainly not. But which is which? Before choosing one of them to ask for directions to platform nine and three-quarters, I have just enough time to ask one of them "Are you a Wizard?", and on the basis of their answer I must make my choice of which guard to ask for directions. I know that a Wizard will answer this question truthfully, but that a Muggle will, with probability $\frac{1}{3}$, answer it untruthfully.

  Failure to catch the Hogwarts Express results in a loss which I measure as 1000 galleons, there being no loss associated with catching up with Harry on the train.

  Write down an exhaustive set of non-randomised decision rules for my problem and, by drawing the associated risk set, determine my minimax decision rule.

  My prior probability is $\frac{2}{3}$ that the guard I ask "Are you a Wizard?" is indeed a Wizard. What is my Bayes decision rule?

  comment

• A2.11 B2.16

  Principles of Statistics

  Part II, 2001

  (i) Let $X_{1}, \ldots, X_{n}$ be independent, identically-distributed $N\left(\mu, \mu^{2}\right)$ random variables, $\mu>0$.

  Find a minimal sufficient statistic for $\mu$.

  Let $T_{1}=n^{-1} \sum_{i=1}^{n} X_{i}$ and $T_{2}=\sqrt{n^{-1} \sum_{i=1}^{n} X_{i}^{2}}$. Write down the distribution of $X_{i} / \mu$, and hence show that $Z=T_{1} / T_{2}$ is ancillary. Explain briefly why the Conditionality Principle would lead to inference about $\mu$ being drawn from the conditional distribution of $T_{2}$ given $Z$.

  What is the maximum likelihood estimator of $\mu$ ?

  (ii) Describe briefly the Bayesian approach to predictive inference,

  Let $Z_{1}, \ldots, Z_{n}$ be independent, identically-distributed $N\left(\mu, \sigma^{2}\right)$ random variables, with $\mu, \sigma^{2}$ both unknown. Derive the maximum likelihood estimators $\widehat{\mu}, \widehat{\sigma}^{2}$ of $\mu, \sigma^{2}$ based on $Z_{1}, \ldots, Z_{n}$, and state, without proof, their joint distribution.

  Suppose that it is required to construct a prediction interval

  $I_{1-\alpha} \equiv I_{1-\alpha}\left(Z_{1}, \ldots, Z_{n}\right)$ for a future, independent, random variable $Z_{0}$ with the same $N\left(\mu, \sigma^{2}\right)$ distribution, such that

  $$P\left(Z_{0} \in I_{1-\alpha}\right)=1-\alpha$$

  with the probability over the joint distribution of $Z_{0}, Z_{1}, \ldots, Z_{n}$. Let

  $$I_{1-\alpha}\left(Z_{1}, \ldots, Z_{n} ; \sigma^{2}\right)=\left[\bar{Z}_{n}-z_{\alpha / 2} \sigma \sqrt{1+1 / n}, \bar{Z}_{n}+z_{\alpha / 2} \sigma \sqrt{1+1 / n}\right]$$

  where $\bar{Z}_{n}=n^{-1} \sum_{i=1}^{n} Z_{i}$, and $\Phi\left(z_{\beta}\right)=1-\beta$, with $\Phi$ the distribution function of $N(0,1)$.

  Show that $P\left(Z_{0} \in I_{1-\alpha}\left(Z_{1}, \ldots, Z_{n} ; \sigma^{2}\right)\right)=1-\alpha$.

  By considering the distribution of $\left(Z_{0}-\bar{Z}_{n}\right) /\left(\widehat{\sigma} \sqrt{\frac{n+1}{n-1}}\right)$, or otherwise, show that

  $$P\left(Z_{0} \in I_{1-\alpha}\left(Z_{1}, \ldots, Z_{n} ; \widehat{\sigma}^{2}\right)\right)<1-\alpha,$$

  and show how to construct an interval $I_{1-\gamma}\left(Z_{1}, \ldots, Z_{n} ; \widehat{\sigma}^{2}\right)$ with

  $$P\left(Z_{0} \in I_{1-\gamma}\left(Z_{1}, \ldots, Z_{n} ; \widehat{\sigma}^{2}\right)\right)=1-\alpha .$$

  [Hint: if $Y$ has the $t$-distribution with $m$ degrees of freedom and $t_{\beta}^{(m)}$ is defined by $P\left(Y<t_{\beta}^{(m)}\right)=1-\beta$ then $t_{\beta}>z_{\beta}$ for $\beta<\frac{1}{2}$.]

  comment

• A3.12 B3.15

  Principles of Statistics

  Part II, 2001

  (i) Explain what is meant by a uniformly most powerful unbiased test of a null hypothesis against an alternative.

  Let $Y_{1}, \ldots, Y_{n}$ be independent, identically distributed $N\left(\mu, \sigma^{2}\right)$ random variables, with $\sigma^{2}$ known. Explain how to construct a uniformly most powerful unbiased size $\alpha$ test of the null hypothesis that $\mu=0$ against the alternative that $\mu \neq 0$.

  (ii) Outline briefly the Bayesian approach to hypothesis testing based on Bayes factors.

  Let the distribution of $Y_{1}, \ldots, Y_{n}$ be as in (i) above, and suppose we wish to test, as in (i), $\mu=0$ against the alternative $\mu \neq 0$. Suppose we assume a $N\left(0, \tau^{2}\right)$ prior for $\mu$ under the alternative. Find the form of the Bayes factor $B$, and show that, for fixed $n, B$ $\rightarrow \infty$ as $\tau \rightarrow \infty$.

  comment

• A4.13 B4.15

  Principles of Statistics

  Part II, 2001

  Write an account, with appropriate examples, of one of the following:

  (a) Inference in multi-parameter exponential families;

  (b) Asymptotic properties of maximum-likelihood estimators and their use in hypothesis testing;

  (c) Bootstrap inference.

  comment

• A1.12 B1.15

  Principles of Statistics

  Part II, 2002

  (i) Explain in detail the minimax and Bayes principles of decision theory.

  Show that if $d(X)$ is a Bayes decision rule for a prior density $\pi(\theta)$ and has constant risk function, then $d(X)$ is minimax.

  (ii) Let $X_{1}, \ldots, X_{p}$ be independent random variables, with $X_{i} \sim N\left(\mu_{i}, 1\right), i=1, \ldots, p$.

  Consider estimating $\mu=\left(\mu_{1}, \ldots, \mu_{p}\right)^{T}$ by $d=\left(d_{1}, \ldots, d_{p}\right)^{T}$, with loss function

  $$L(\mu, d)=\sum_{i=1}^{p}\left(\mu_{i}-d_{i}\right)^{2}$$

  What is the risk function of $X=\left(X_{1}, \ldots, X_{p}\right)^{T} ?$

  Consider the class of estimators of $\mu$ of the form

  $$d^{a}(X)=\left(1-\frac{a}{X^{T} X}\right) X$$

  indexed by $a \geqslant 0$. Find the risk function of $d^{a}(X)$ in terms of $E\left(1 / X^{T} X\right)$, which you should not attempt to evaluate, and deduce that $X$ is inadmissible. What is optimal value of $a$ ?

  [You may assume Stein's Lemma, that for suitably behaved real-valued functions $h$,

  $$\left.E\left\{\left(X_{i}-\mu_{i}\right) h(X)\right\}=E\left\{\frac{\partial h(X)}{\partial X_{i}}\right\} . \quad\right]$$

  comment

• A2.11 B2.16

  Principles of Statistics

  Part II, 2002

  (i) Let $X$ be a random variable with density function $f(x ; \theta)$. Consider testing the simple null hypothesis $H_{0}: \theta=\theta_{0}$ against the simple alternative hypothesis $H_{1}: \theta=\theta_{1}$.

  What is the form of the optimal size $\alpha$ classical hypothesis test?

  Compare the form of the test with the Bayesian test based on the Bayes factor, and with the Bayes decision rule under the 0-1 loss function, under which a loss of 1 is incurred for an incorrect decision and a loss of 0 is incurred for a correct decision.

  (ii) What does it mean to say that a family of densities $\{f(x ; \theta), \theta \in \Theta\}$ with real scalar parameter $\theta$ is of monotone likelihood ratio?

  Suppose $X$ has a distribution from a family which is of monotone likelihood ratio with respect to a statistic $t(X)$ and that it is required to test $H_{0}: \theta \leqslant \theta_{0}$ against $H_{1}: \theta>\theta_{0}$.

  State, without proof, a theorem which establishes the existence of a uniformly most powerful test and describe in detail the form of the test.

  Let $X_{1}, \ldots, X_{n}$ be independent, identically distributed $U(0, \theta), \theta>0$. Find a uniformly most powerful size $\alpha$ test of $H_{0}: \theta \leqslant \theta_{0}$ against $H_{1}: \theta>\theta_{0}$, and find its power function. Show that we may construct a different, randomised, size $\alpha$ test with the same power function for $\theta \geqslant \theta_{0}$.

  comment

• A3.12 B3.15

  Principles of Statistics

  Part II, 2002

  (i) Describe in detail how to perform the Wald, score and likelihood ratio tests of a simple null hypothesis $H_{0}: \theta=\theta_{0}$ given a random sample $X_{1}, \ldots, X_{n}$ from a regular oneparameter density $f(x ; \theta)$. In each case you should specify the asymptotic null distribution of the test statistic.

  (ii) Let $X_{1}, \ldots, X_{n}$ be an independent, identically distributed sample from a distribution $F$, and let $\hat{\theta}\left(X_{1}, \ldots, X_{n}\right)$ be an estimator of a parameter $\theta$ of $F$.

  Explain what is meant by: (a) the empirical distribution function of the sample; (b) the bootstrap estimator of the bias of $\hat{\theta}$, based on the empirical distribution function. Explain how a bootstrap estimator of the distribution function of $\hat{\theta}-\theta$ may be used to construct an approximate $1-\alpha$ confidence interval for $\theta$.

  Suppose the parameter of interest is $\theta=\mu^{2}$, where $\mu$ is the mean of $F$, and the estimator is $\hat{\theta}=\bar{X}^{2}$, where $\bar{X}=n^{-1} \sum_{i=1}^{n} X_{i}$ is the sample mean.

  Derive an explicit expression for the bootstrap estimator of the bias of $\hat{\theta}$ and show that it is biased as an estimator of the true bias of $\hat{\theta}$.

  Let $\hat{\theta}_{i}$ be the value of the estimator $\hat{\theta}\left(X_{1}, \ldots, X_{i-1}, X_{i+1}, \ldots, X_{n}\right)$ computed from the sample of size $n-1$ obtained by deleting $X_{i}$ and let $\hat{\theta} . n^{-1} \sum_{i=1}^{n} \hat{\theta}_{i}$. The jackknife estimator of the bias of $\hat{\theta}$ is

  $$b_{J}=(n-1)(\hat{\theta} .-\hat{\theta}) .$$

  Derive the jackknife estimator $b_{J}$ for the case $\hat{\theta}=\bar{X}^{2}$, and show that, as an estimator of the true bias of $\hat{\theta}$, it is unbiased.

  comment

• A4.13 B4.15

  Principles of Statistics

  Part II, 2002

  (a) Let $X_{1}, \ldots, X_{n}$ be independent, identically distributed random variables from a one-parameter distribution with density function

  $$f(x ; \theta)=h(x) g(\theta) \exp \{\theta t(x)\}, x \in \mathbb{R} .$$

  Explain in detail how you would test

  $$H_{0}: \theta=\theta_{0} \text { against } H_{1}: \theta \neq \theta_{0} \text {. }$$

  What is the general form of a conjugate prior density for $\theta$ in a Bayesian analysis of this distribution?

  (b) Let $Y_{1}, Y_{2}$ be independent Poisson random variables, with means $(1-\psi) \lambda$ and $\psi \lambda$ respectively, with $\lambda$ known.

  Explain why the Conditionality Principle leads to inference about $\psi$ being drawn from the conditional distribution of $Y_{2}$, given $Y_{1}+Y_{2}$. What is this conditional distribution?

  (c) Suppose $Y_{1}, Y_{2}$ have distributions as in (b), but that $\lambda$ is now unknown.

  Explain in detail how you would test $H_{0}: \psi=\psi_{0}$ against $H_{1}: \psi \neq \psi_{0}$, and describe the optimality properties of your test.

  [Any general results you use should be stated clearly, but need not be proved.]

  comment

• A1.12 B1.15

  Principles of Statistics

  Part II, 2003

  (i) A public health official is seeking a rational policy of vaccination against a relatively mild ailment which causes absence from work. Surveys suggest that $60 \%$ of the population are already immune, but accurate tests to detect vulnerability in any individual are too costly for mass screening. A simple skin test has been developed, but is not completely reliable. A person who is immune to the ailment will have a negligible reaction to the skin test with probability $0.4$, a moderate reaction with probability $0.5$ and a strong reaction with probability 0.1. For a person who is vulnerable to the ailment the corresponding probabilities are $0.1,0.4$ and $0.5$. It is estimated that the money-equivalent of workhours lost from failing to vaccinate a vulnerable person is 20 , that the unnecessary cost of vaccinating an immune person is 8 , and that there is no cost associated with vaccinating a vulnerable person or failing to vaccinate an immune person. On the basis of the skin test, it must be decided whether to vaccinate or not. What is the Bayes decision rule that the health official should adopt?

  (ii) A collection of $I$ students each sit $J$ exams. The ability of the $i$ th student is represented by $\theta_{i}$ and the performance of the $i$ th student on the $j$ th exam is measured by $X_{i j}$. Assume that, given $\boldsymbol{\theta}=\left(\theta_{1}, \ldots, \theta_{I}\right)$, an appropriate model is that the variables $\left\{X_{i j}, 1 \leqslant i \leqslant I, 1 \leqslant j \leqslant J\right\}$ are independent, and

  $$X_{i j} \sim N\left(\theta_{i}, \tau^{-1}\right),$$

  for a known positive constant $\tau$. It is reasonable to assume, a priori, that the $\theta_{i}$ are independent with

  $$\theta_{i} \sim N\left(\mu, \zeta^{-1}\right),$$

  where $\mu$ and $\zeta$ are population parameters, known from experience with previous cohorts of students.

  Compute the posterior distribution of $\boldsymbol{\theta}$ given the observed exam marks vector $\mathbf{X}=\left\{X_{i j}, 1 \leqslant i \leqslant I, 1 \leqslant j \leqslant J\right\} .$

  Suppose now that $\tau$ is also unknown, but assumed to have a $\operatorname{Gamma}\left(\alpha_{0}, \beta_{0}\right)$ distribution, for known $\alpha_{0}, \beta_{0}$. Compute the posterior distribution of $\tau$ given $\boldsymbol{\theta}$ and $\mathbf{X}$ Find, up to a normalisation constant, the form of the marginal density of $\boldsymbol{\theta}$ given $\mathbf{X}$.

  comment

• A2.11 B2.16

  Principles of Statistics

  Part II, 2003

  (i) Outline briefly the Bayesian approach to hypothesis testing based on Bayes factors.

  (ii) Let $Y_{1}, Y_{2}$ be independent random variables, both uniformly distributed on $\left(\theta-\frac{1}{2}, \theta+\frac{1}{2}\right)$. Find a minimal sufficient statistic for $\theta$. Let $Y_{(1)}=\min \left\{Y_{1}, Y_{2}\right\}$, $Y_{(2)}=\max \left\{Y_{1}, Y_{2}\right\}$. Show that $R=Y_{(2)}-Y_{(1)}$ is ancilliary and explain why the Conditionality Principle would lead to inference about $\theta$ being drawn from the conditional distribution of $\frac{1}{2}\left\{Y_{(1)}+Y_{(2)}\right\}$ given $R$. Find the form of this conditional distribution.

  comment

• A3.12 B3.15

  Principles of Statistics

  Part II, 2003

  (i) Let $X_{1}, \ldots, X_{n}$ be independent, identically distributed random variables, with the exponential density $f(x ; \theta)=\theta e^{-\theta x}, x>0$.

  Obtain the maximum likelihood estimator $\hat{\theta}$ of $\theta$. What is the asymptotic distribution of $\sqrt{n}(\hat{\theta}-\theta)$ ?

  What is the minimum variance unbiased estimator of $\theta ?$ Justify your answer carefully.

  (ii) Explain briefly what is meant by the profile log-likelihood for a scalar parameter of interest $\gamma$, in the presence of a nuisance parameter $\xi$. Describe how you would test a null hypothesis of the form $H_{0}: \gamma=\gamma_{0}$ using the profile log-likelihood ratio statistic.

  In a reliability study, lifetimes $T_{1}, \ldots, T_{n}$ are independent and exponentially distributed, with means of the form $E\left(T_{i}\right)=\exp \left(\beta+\xi z_{i}\right)$ where $\beta, \xi$ are unknown and $z_{1}, \ldots, z_{n}$ are known constants. Inference is required for the mean lifetime, $\exp \left(\beta+\xi z_{0}\right)$, for covariate value $z_{0}$.

  Find, as explicitly as possible, the profile log-likelihood for $\gamma \equiv \beta+\xi z_{0}$, with nuisance parameter $\xi$.

  Show that, under $H_{0}: \gamma=\gamma_{0}$, the profile $\log -$ likelihood ratio statistic has a distribution which does not depend on the value of $\xi$. How might the parametric bootstrap be used to obtain a test of $H_{0}$ of exact size $\alpha$ ?

  [Hint: if $Y$ is exponentially distributed with mean 1 , then $\mu Y$ is exponentially distributed with mean $\mu$.]

  comment

• A4.13 B4.15

  Principles of Statistics

  Part II, 2003

  Write an account, with appropriate examples, of inference in multiparameter exponential families. Your account should include a discussion of natural statistics and their properties and of various conditional tests on natural parameters.

  comment

• A1.12 B1.15

  Principles of Statistics

  Part II, 2004

  (i) What does it mean to say that a family $\{f(\cdot \mid \theta): \theta \in \Theta\}$ of densities is an exponential family?

  Consider the family of densities on $(0, \infty)$ parametrised by the positive parameters $a, b$ and defined by

  $$f(x \mid a, b)=\frac{a \exp \left(-(a-b x)^{2} / 2 x\right)}{\sqrt{2 \pi x^{3}}} \quad(x>0)$$

  Prove that this family is an exponential family, and identify the natural parameters and the reference measure.

  (ii) Let $\left(X_{1}, \ldots, X_{n}\right)$ be a sample drawn from the above distribution. Find the maximum-likelihood estimators of the parameters $(a, b)$. Find the Fisher information matrix of the family (in terms of the natural parameters). Briefly explain the significance of the Fisher information matrix in relation to unbiased estimation. Compute the mean of $X_{1}$ and of $X_{1}^{-1}$.

  comment

• A2.11 B2.16

  Principles of Statistics

  Part II, 2004

  (i) In the context of a decision-theoretic approach to statistics, what is a loss function? a decision rule? the risk function of a decision rule? the Bayes risk of a decision rule? the Bayes rule with respect to a given prior distribution?

  Show how the Bayes rule with respect to a given prior distribution is computed.

  (ii) A sample of $n$ people is to be tested for the presence of a certain condition. A single real-valued observation is made on each one; this observation comes from density $f_{0}$ if the condition is absent, and from density $f_{1}$ if the condition is present. Suppose $\theta_{i}=0$ if the $i^{\text {th }}$person does not have the condition, $\theta_{i}=1$ otherwise, and suppose that the prior distribution for the $\theta_{i}$ is that they are independent with common distribution $P\left(\theta_{i}=1\right)=p \in(0,1)$, where $p$ is known. If $X_{i}$ denotes the observation made on the $i^{\text {th }}$person, what is the posterior distribution of the $\theta_{i}$ ?

  Now suppose that the loss function is defined by

  $$L_{0}(\theta, a) \equiv \sum_{j=1}^{n}\left(\alpha a_{j}\left(1-\theta_{j}\right)+\beta\left(1-a_{j}\right) \theta_{j}\right)$$

  for action $a \in[0,1]^{n}$, where $\alpha, \beta$ are positive constants. If $\pi_{j}$ denotes the posterior probability that $\theta_{j}=1$ given the data, prove that the Bayes rule for this prior and this loss function is to take $a_{j}=1$ if $\pi_{j}$ exceeds the threshold value $\alpha /(\alpha+\beta)$, and otherwise to take $a_{j}=0$.

  In an attempt to control the proportion of false positives, it is proposed to use a different loss function, namely,

  $$L_{1}(\theta, a) \equiv L_{0}(\theta, a)+\gamma I_{\left\{\sum a_{j}>0\right\}}\left(1-\frac{\sum \theta_{j} a_{j}}{\sum a_{j}}\right)$$

  where $\gamma>0$. Prove that the Bayes rule is once again a threshold rule, that is, we take action $a_{j}=1$ if and only if $\pi_{j}>\lambda$, and determine $\lambda$ as fully as you can.

  comment

• A3.12 B3.15

  Principles of Statistics

  Part II, 2004

  (i) What is a sufficient statistic? What is a minimal sufficient statistic? Explain the terms nuisance parameter and ancillary statistic.

  (ii) Let $U_{1}, \ldots, U_{n}$ be independent random variables with common uniform( $([0,1])$ distribution, and suppose you observe $X_{i} \equiv a U_{i}^{-\beta}, i=1, \ldots, n$, where the positive parameters $a, \beta$ are unknown. Write down the joint density of $X_{1}, \ldots, X_{n}$ and prove that the statistic

  $$(m, p) \equiv\left(\min _{1 \leqslant j \leqslant n}\left\{X_{j}\right\}, \prod_{j=1}^{n} X_{j}\right)$$

  is minimal sufficient for $(a, \beta)$. Find the maximum-likelihood estimator $(\hat{a}, \hat{\beta})$ of $(a, \beta)$.

  Regarding $\beta$ as the parameter of interest and $a$ as the nuisance parameter, is $m$ ancillary? Find the mean and variance of $\hat{\beta}$. Hence find an unbiased estimator of $\beta$.

  comment

• A4.13 B4.15

  Principles of Statistics

  Part II, 2004

  Suppose that $\theta \in \mathbb{R}^{d}$ is the parameter of a non-degenerate exponential family. Derive the asymptotic distribution of the maximum-likelihood estimator $\hat{\theta}_{n}$ of $\theta$ based on a sample of size $n$. [You may assume that the density is infinitely differentiable with respect to the parameter, and that differentiation with respect to the parameter commutes with integration.]

  Part II 2004

  comment

• 1.II.27I

  Principles of Statistics

  Part II, 2005

  State Wilks' Theorem on the asymptotic distribution of likelihood-ratio test statistics.

  Suppose that $X_{1}, \ldots, X_{n}$ are independent with common $N\left(\mu, \sigma^{2}\right)$ distribution, where the parameters $\mu$ and $\sigma$ are both unknown. Find the likelihood-ratio test statistic for testing $H_{0}: \mu=0$ against $H_{1}: \mu$ unrestricted, and state its (approximate) distribution.

  What is the form of the $t$-test of $H_{0}$ against $H_{1}$ ? Explain why for large $n$ the likelihood-ratio test and the $t$-test are nearly the same.

  comment

• 2.II.27I

  Principles of Statistics

  Part II, 2005

  (i) Suppose that $X$ is a multivariate normal vector with mean $\mu \in \mathbb{R}^{d}$ and covariance matrix $\sigma^{2} I$, where $\mu$ and $\sigma^{2}$ are both unknown, and $I$ denotes the $d \times d$ identity matrix. Suppose that $\Theta_{0} \subset \Theta_{1}$ are linear subspaces of $\mathbb{R}^{d}$ of dimensions $d_{0}$ and $d_{1}$, where $d_{0}<d_{1}<d$. Let $P_{i}$ denote orthogonal projection onto $\Theta_{i}(i=0,1)$. Carefully derive the joint distribution of $\left(\left|X-P_{1} X\right|^{2},\left|P_{1} X-P_{0} X\right|^{2}\right)$ under the hypothesis $H_{0}: \mu \in \Theta_{0}$. How could you use this to make a test of $H_{0}$ against $H_{1}: \mu \in \Theta_{1}$ ?

  (ii) Suppose that $I$ students take $J$ exams, and that the mark $X_{i j}$ of student $i$ in exam $j$ is modelled as

  $$X_{i j}=m+\alpha_{i}+\beta_{j}+\varepsilon_{i j}$$

  where $\sum_{i} \alpha_{i}=0=\sum_{j} \beta_{j}$, the $\varepsilon_{i j}$ are independent $N\left(0, \sigma^{2}\right)$, and the parameters $m, \alpha, \beta$ and $\sigma$ are unknown. Construct a test of $H_{0}: \beta_{j}=0$ for all $j$ against $H_{1}: \sum_{j} \beta_{j}=0$.

  comment

• 3.II.26I

  Principles of Statistics

  Part II, 2005

  In the context of decision theory, explain the meaning of the following italicized terms: loss function, decision rule, the risk of a decision rule, a Bayes rule with respect to prior $\pi$, and an admissible rule. Explain how a Bayes rule with respect to a prior $\pi$ can be constructed.

  Suppose that $X_{1}, \ldots, X_{n}$ are independent with common $N(0, v)$ distribution, where $v>0$ is supposed to have a prior density $f_{0}$. In a decision-theoretic approach to estimating $v$, we take a quadratic loss: $L(v, a)=(v-a)^{2}$. Write $X=\left(X_{1}, \ldots, X_{n}\right)$ and $|X|=\left(X_{1}^{2}+\ldots+X_{n}^{2}\right)^{1 / 2}$.

  By considering decision rules (estimators) of the form $\hat{v}(X)=\alpha|X|^{2}$, prove that if $\alpha \neq 1 /(n+2)$ then the estimator $\hat{v}(X)=\alpha|X|^{2}$ is not Bayes, for any choice of prior $f_{0}$.

  By considering decision rules of the form $\hat{v}(X)=\alpha|X|^{2}+\beta$, prove that if $\alpha \neq 1 / n$ then the estimator $\hat{v}(X)=\alpha|X|^{2}$ is not Bayes, for any choice of prior $f_{0}$.

  [You may use without proof the fact that, if $Z$ has a $N(0,1)$ distribution, then $E Z^{4}=3$.]

  comment

• 4.II.27I

  Principles of Statistics

  Part II, 2005

  A group of $n$ hospitals is to be 'appraised'; the 'performance' $\theta_{i}$ of hospital $i$ has a $N(0,1 / \tau)$ prior distribution, different hospitals being independent. The 'performance' cannot be measured directly, so an expensive firm of management consultants has been hired to arrive at each hospital's Standardised Index of Quality [SIQ], this being a number $X_{i}$ for hospital $i$ related to $\theta_{i}$ by the commercially-sensitive formula

  $$X_{i}=\theta_{i}+\varepsilon_{i}$$

  where the $\varepsilon_{i}$ are independent with common $N\left(0,1 / \tau_{\varepsilon}\right)$ distribution.

  (i) Assume that $\tau$ and $\tau_{\varepsilon}$ are known. What is the posterior distribution of $\theta$ given $X$ ? Suppose that hospital $j$ was the hospital with the lowest SIQ, with a value $X_{j}=x$; conditional on $X$, what is the distribution of $\theta_{j}$ ?

  (ii) Now, instead of assuming $\tau$ and $\tau_{\varepsilon}$ known, suppose that $\tau$ has a Gamma prior with parameters $(\alpha, \beta)$, density

  $$f(t)=(\beta t)^{\alpha-1} \beta e^{-\beta t} / \Gamma(\alpha)$$

  for known $\alpha$ and $\beta$, and that $\tau_{\varepsilon}=\kappa \tau$, where $\kappa$ is a known constant. Find the posterior distribution of $(\theta, \tau)$ given $X$. Comment briefly on the form of the distribution.

  comment

• 1.II $. 27 \mathrm{~J} \quad$

  Principles of Statistics

  Part II, 2006

  (a) What is a loss function? What is a decision rule? What is the risk function of a decision rule? What is the Bayes risk of a decision rule with respect to a prior $\pi$ ?

  (b) Let $\theta \mapsto R(\theta, d)$ denote the risk function of decision rule $d$, and let $r(\pi, d)$ denote the Bayes risk of decision rule $d$ with respect to prior $\pi$. Suppose that $d^{*}$ is a decision rule and $\pi_{0}$ is a prior over the parameter space $\Theta$ with the two properties

  (i) $r\left(\pi_{0}, d^{*}\right)=\min _{d} r\left(\pi_{0}, d\right)$

  (ii) $\sup _{\theta} R\left(\theta, d^{*}\right)=r\left(\pi_{0}, d^{*}\right)$.

  Prove that $d^{*}$ is minimax.

  (c) Suppose now that $\Theta=\mathcal{A}=\mathbb{R}$, where $\mathcal{A}$ is the space of possible actions, and that the loss function is

  $$L(\theta, a)=\exp (-\lambda a \theta),$$

  where $\lambda$ is a positive constant. If the law of the observation $X$ given parameter $\theta$ is $N\left(\theta, \sigma^{2}\right)$, where $\sigma>0$ is known, show (using (b) or otherwise) that the rule

  $$d^{*}(x)=x / \sigma^{2} \lambda$$

  is minimax.

  comment

• 2.II.27J

  Principles of Statistics

  Part II, 2006

  Let $\{f(\cdot \mid \theta): \theta \in \Theta\}$ be a parametric family of densities for observation $X$. What does it mean to say that the statistic $T \equiv T(X)$ is sufficient for $\theta$ ? What does it mean to say that $T$ is minimal sufficient?

  State the Rao-Blackwell theorem. State the Cramér-Rao lower bound for the variance of an unbiased estimator of a (scalar) parameter, taking care to specify any assumptions needed.

  Let $X_{1}, \ldots, X_{n}$ be a sample from a $U(0, \theta)$ distribution, where the positive parameter $\theta$ is unknown. Find a minimal sufficient statistic $T$ for $\theta$. If $h(T)$ is an unbiased estimator for $\theta$, find the form of $h$, and deduce that this estimator is minimum-variance unbiased. Would it be possible to reach this conclusion using the Cramér-Rao lower bound?

  comment

• 3.II.26J

  Principles of Statistics

  Part II, 2006

  Write an essay on the rôle of the Metropolis-Hastings algorithm in computational Bayesian inference on a parametric model. You may for simplicity assume that the parameter space is finite. Your essay should:

  (a) explain what problem in Bayesian inference the Metropolis-Hastings algorithm is used to tackle;

  (b) fully justify that the algorithm does indeed deliver the required information about the model;

  (c) discuss any implementational issues that need care.

  comment

• 4.II.27J

  Principles of Statistics

  Part II, 2006

  (a) State the strong law of large numbers. State the central limit theorem.

  (b) Assuming whatever regularity conditions you require, show that if $\hat{\theta}_{n} \equiv \hat{\theta}_{n}\left(X_{1}, \ldots, X_{n}\right)$ is the maximum-likelihood estimator of the unknown parameter $\theta$ based on an independent identically distributed sample of size $n$, then under $P_{\theta}$

  $$\sqrt{n}\left(\hat{\theta}_{n}-\theta\right) \rightarrow N\left(0, J(\theta)^{-1}\right) \quad \text { in distribution }$$

  as $n \rightarrow \infty$, where $J(\theta)$ is a matrix which you should identify. A rigorous derivation is not required.

  (c) Suppose that $X_{1}, X_{2}, \ldots$ are independent binomial $\operatorname{Bin}(1, \theta)$ random variables. It is required to test $H_{0}: \theta=\frac{1}{2}$ against the alternative $H_{1}: \theta \in(0,1)$. Show that the construction of a likelihood-ratio test leads us to the statistic

  $$T_{n}=2 n\left\{\hat{\theta}_{n} \log \hat{\theta}_{n}+\left(1-\hat{\theta}_{n}\right) \log \left(1-\hat{\theta}_{n}\right)+\log 2\right\}$$

  where $\hat{\theta}_{n} \equiv n^{-1} \sum_{k=1}^{n} X_{k}$. Stating clearly any result to which you appeal, for large $n$, what approximately is the distribution of $T_{n}$ under $H_{0}$ ? Writing $\hat{\theta}_{n}=\frac{1}{2}+Z_{n}$, and assuming that $Z_{n}$ is small, show that

  $$T_{n} \simeq 4 n Z_{n}^{2}$$

  Using this and the central limit theorem, briefly justify the approximate distribution of $T_{n}$ given by asymptotic maximum-likelihood theory. What could you say if the assumption that $Z_{n}$ is small failed?

  comment

• 1.II.27I

  Principles of Statistics

  Part II, 2007

  Suppose that $X$ has density $f(\cdot \mid \theta)$ where $\theta \in \Theta$. What does it mean to say that statistic $T \equiv T(X)$ is sufficient for $\theta$ ?

  Suppose that $\theta=(\psi, \lambda)$, where $\psi$ is the parameter of interest, and $\lambda$ is a nuisance parameter, and that the sufficient statistic $T$ has the form $T=(C, S)$. What does it mean to say that the statistic $S$ is ancillary? If it is, how (according to the conditionality principle) do we test hypotheses on $\psi ?$ Assuming that the set of possible values for $X$ is discrete, show that $S$ is ancillary if and only if the density (probability mass function) $f(x \mid \psi, \lambda)$ factorises as

  $$f(x \mid \psi, \lambda)=\varphi_{0}(x) \varphi_{C}(C(x), S(x), \psi) \varphi_{S}(S(x), \lambda)$$

  for some functions $\varphi_{0}, \varphi_{C}$, and $\varphi_{S}$ with the properties

  $$\sum_{x \in C^{-1}(c) \cap S^{-1}(s)} \varphi_{0}(x)=1=\sum_{s} \varphi_{S}(s, \lambda)=\sum_{s} \sum_{c} \varphi_{C}(c, s, \psi)$$

  for all $c, s, \psi$, and $\lambda$.

  Suppose now that $X_{1}, \ldots, X_{n}$ are independent observations from a $\Gamma(a, b)$ distribution, with density

  $$f(x \mid a, b)=(b x)^{a-1} e^{-b x} b I_{\{x>0\}} / \Gamma(a) .$$

  Assuming that the criterion (*) holds also for observations which are not discrete, show that it is not possible to find $(C(X), S(X))$ sufficient for $(a, b)$ such that $S$ is ancillary when $b$ is regarded as a nuisance parameter, and $a$ is the parameter of interest.

  comment

• 2.II.27I

  Principles of Statistics

  Part II, 2007

  (i) State Wilks' likelihood ratio test of the null hypothesis $H_{0}: \theta \in \Theta_{0}$ against the alternative $H_{1}: \theta \in \Theta_{1}$, where $\Theta_{0} \subset \Theta_{1}$. Explain when this test may be used.

  (ii) Independent identically-distributed observations $X_{1}, \ldots, X_{n}$ take values in the set $S=\{1, \ldots, K\}$, with common distribution which under the null hypothesis is of the form

  $$P\left(X_{1}=k \mid \theta\right)=f(k \mid \theta) \quad(k \in S)$$

  for some $\theta \in \Theta_{0}$, where $\Theta_{0}$ is an open subset of some Euclidean space $\mathbb{R}^{d}$, $d<K-1$. Under the alternative hypothesis, the probability mass function of the $X_{i}$ is unrestricted.

  Assuming sufficient regularity conditions on $f$ to guarantee the existence and uniqueness of a maximum-likelihood estimator $\hat{\theta}_{n}\left(X_{1}, \ldots, X_{n}\right)$ of $\theta$ for each $n$, show that for large $n$ the Wilks' likelihood ratio test statistic is approximately of the form

  $$\sum_{j=1}^{K}\left(N_{j}-n \hat{\pi}_{j}\right)^{2} / N_{j}$$

  where $N_{j}=\sum_{i=1}^{n} I_{\left\{X_{i}=j\right\}}$, and $\hat{\pi}_{j}=f\left(j \mid \hat{\theta}_{n}\right)$. What is the asymptotic distribution of this statistic?

  comment

• 3.II.26I

  Principles of Statistics

  Part II, 2007

  (i) In the context of decision theory, what is a Bayes rule with respect to a given loss function and prior? What is an extended Bayes rule?

  Characterise the Bayes rule with respect to a given prior in terms of the posterior distribution for the parameter given the observation. When $\Theta=\mathcal{A}=\mathbb{R}^{d}$ for some $d$, and the loss function is $L(\theta, a)=\|\theta-a\|^{2}$, what is the Bayes rule?

  (ii) Suppose that $\mathcal{A}=\Theta=\mathbb{R}$, with loss function $L(\theta, a)=(\theta-a)^{2}$ and suppose further that under $P_{\theta}, X \sim N(\theta, 1)$.

  Supposing that a $N\left(0, \tau^{-1}\right)$ prior is taken over $\theta$, compute the Bayes risk of the decision rule $d_{\lambda}(X)=\lambda X$. Find the posterior distribution of $\theta$ given $X$, and confirm that its mean is of the form $d_{\lambda}(X)$ for some value of $\lambda$ which you should identify. Hence show that the decision rule $d_{1}$ is an extended Bayes rule.

  comment

• 4.II.27I

  Principles of Statistics

  Part II, 2007

  Assuming sufficient regularity conditions on the likelihood $f(x \mid \theta)$ for a univariate parameter $\theta \in \Theta$, establish the Cramér-Rao lower bound for the variance of an unbiased estimator of $\theta$.

  If $\hat{\theta}(X)$ is an unbiased estimator of $\theta$ whose variance attains the Cramér-Rao lower bound for every value of $\theta \in \Theta$, show that the likelihood function is an exponential family.

  comment

• 1.II.27I

  Principles of Statistics

  Part II, 2008

  An angler starts fishing at time 0. Fish bite in a Poisson Process of rate $\Lambda$ per hour, so that, if $\Lambda=\lambda$, the number $N_{t}$ of fish he catches in the first $t$ hours has the Poisson distribution $\mathcal{P}(\lambda t)$, while $T_{n}$, the time in hours until his $n$th bite, has the Gamma distribution $\Gamma(n, \lambda)$, with density function

  $$p(t \mid \lambda)=\frac{\lambda^{n}}{(n-1) !} t^{n-1} e^{-\lambda t} \quad(t>0) .$$

  Bystander $B_{1}$ plans to watch for 3 hours, and to record the number $N_{3}$ of fish caught. Bystander $B_{2}$ plans to observe until the 10 th bite, and to record $T_{10}$, the number of hours until this occurs.

  For $B_{1}$, show that $\widetilde{\Lambda}_{1}:=N_{3} / 3$ is an unbiased estimator of $\Lambda$ whose variance function achieves the Cramér-Rao lower bound

  Find an unbiased estimator of $\Lambda$ for $B_{2}$, of the form $\widetilde{\Lambda}_{2}=k / T_{10}$. Does it achieve the Cramér-Rao lower bound? Is it minimum-variance-unbiased? Justify your answers.

  In fact, the 10 th fish bites after exactly 3 hours. For each of $B_{1}$ and $B_{2}$, write down the likelihood function for $\Lambda$ based their observations. What does the Likelihood Principle have to say about the inferences to be drawn by $B_{1}$ and $B_{2}$, and why? Compute the estimates $\tilde{\lambda}_{1}$ and $\widetilde{\lambda}_{2}$ produced by applying $\widetilde{\Lambda}_{1}$ and $\widetilde{\Lambda}_{2}$ to the observed data. Does the method of minimum-variance-unbiased estimation respect the Likelihood Principle?

  comment

• 2.II.27I

  Principles of Statistics

  Part II, 2008

  Under hypothesis $H_{i}(i=0,1)$, a real-valued observable $X$, taking values in $\mathcal{X}$, has density function $p_{i}(\cdot)$. Define the Type I error $\alpha$ and the Type II error $\beta$ of a test $\phi: \mathcal{X} \rightarrow[0,1]$ of the null hypothesis $H_{0}$ against the alternative hypothesis $H_{1}$. What are the size and power of the test in terms of $\alpha$ and $\beta$ ?

  Show that, for $0<c<\infty, \phi$ minimises $c \alpha+\beta$ among all possible tests if and only if it satisfies

  $$\begin{aligned} &p_{1}(x)>c p_{0}(x) \Rightarrow \phi(x)=1 \\ &p_{1}(x)<c p_{0}(x) \Rightarrow \phi(x)=0 \end{aligned}$$

  What does this imply about the admissibility of such a test?

  Given the value $\theta$ of a parameter variable $\Theta \in[0,1)$, the observable $X$ has density function

  $$p(x \mid \theta)=\frac{2(x-\theta)}{(1-\theta)^{2}} \quad(\theta \leqslant x \leqslant 1)$$

  For fixed $\theta \in(0,1)$, describe all the likelihood ratio tests of $H_{0}: \Theta=0$ against $H_{\theta}: \Theta=\theta$.

  For fixed $k \in(0,1)$, let $\phi_{k}$ be the test that rejects $H_{0}$ if and only if $X \geqslant k$. Is $\phi_{k}$ admissible as a test of $H_{0}$ against $H_{\theta}$ for every $\theta \in(0,1)$ ? Is it uniformly most powerful for its size for testing $H_{0}$ against the composite hypothesis $H_{1}: \Theta \in(0,1)$ ? Is it admissible as a test of $H_{0}$ against $H_{1}$ ?

  comment

• 3.II.26I

  Principles of Statistics

  Part II, 2008

  Define the notion of exponential family $(E F)$, and show that, for data arising as a random sample of size $n$ from an exponential family, there exists a sufficient statistic whose dimension stays bounded as $n \rightarrow \infty$.

  The log-density of a normal distribution $\mathcal{N}(\mu, v)$ can be expressed in the form

  $$\log p(x \mid \boldsymbol{\phi})=\phi_{1} x+\phi_{2} x^{2}-k(\boldsymbol{\phi})$$

  where $\phi=\left(\phi_{1}, \phi_{2}\right)$ is the value of an unknown parameter $\Phi=\left(\Phi_{1}, \Phi_{2}\right)$. Determine the function $k$, and the natural parameter-space $\mathbb{F}$. What is the mean-value parameter $\mathrm{H}=\left(\mathrm{H}_{1}, \mathrm{H}_{2}\right)$ in terms of $\Phi ?$

  Determine the maximum likelihood estimator $\widehat{\Phi}_{1}$ of $\Phi_{1}$ based on a random sample $\left(X_{1}, \ldots, X_{n}\right)$, and give its asymptotic distribution for $n \rightarrow \infty$.

  How would these answers be affected if the variance of $X$ were known to have value $v_{0}$ ?

  comment

• 4.II.27I

  Principles of Statistics

  Part II, 2008

  Define sufficient statistic, and state the factorisation criterion for determining whether a statistic is sufficient. Show that a Bayesian posterior distribution depends on the data only through the value of a sufficient statistic.

  Given the value $\mu$ of an unknown parameter $M$, observables $X_{1}, \ldots, X_{n}$ are independent and identically distributed with distribution $\mathcal{N}(\mu, 1)$. Show that the statistic $\bar{X}:=n^{-1} \sum_{i=1}^{n} X_{i}$ is sufficient for $\mathrm{M}$.

  If the prior distribution is $\mathrm{M} \sim \mathcal{N}\left(0, \tau^{2}\right)$, determine the posterior distribution of $\mathrm{M}$ and the predictive distribution of $\bar{X}$.

  In fact, there are two hypotheses as to the value of M. Under hypothesis $H_{0}$, $\mathrm{M}$ takes the known value 0 ; under $H_{1}, \mathrm{M}$ is unknown, with prior distribution $\mathcal{N}\left(0, \tau^{2}\right)$. Explain why the Bayes factor for choosing between $H_{0}$ and $H_{1}$ depends only on $\bar{X}$, and determine its value for data $X_{1}=x_{1}, \ldots, X_{n}=x_{n}$.

  The frequentist $5 \%$-level test of $H_{0}$ against $H_{1}$ rejects $H_{0}$ when $|\bar{X}| \geqslant 1.96 / \sqrt{n}$. What is the Bayes factor for the critical case $|\bar{x}|=1.96 / \sqrt{n}$ ? How does this behave as $n \rightarrow \infty$ ? Comment on the similarities or differences in behaviour between the frequentist and Bayesian tests.

  comment

• Paper 3, Section II, I

  Principles of Statistics

  Part II, 2009

  What is meant by an equaliser decision rule? What is meant by an extended Bayes rule? Show that a decision rule that is both an equaliser rule and extended Bayes is minimax.

  Let $X_{1}, \ldots, X_{n}$ be independent and identically distributed random variables with the normal distribution $\mathcal{N}\left(\theta, h^{-1}\right)$, and let $k>0$. It is desired to estimate $\theta$ with loss function $L(\theta, a)=1-\exp \left\{-\frac{1}{2} k(a-\theta)^{2}\right\}$.

  Suppose the prior distribution is $\theta \sim \mathcal{N}\left(m_{0}, h_{0}^{-1}\right)$. Find the Bayes act and the Bayes loss posterior to observing $X_{1}=x_{1}, \ldots, X_{n}=x_{n}$. What is the Bayes risk of the Bayes rule with respect to this prior distribution?

  Show that the rule that estimates $\theta$ by $\bar{X}=n^{-1} \sum_{i=1}^{n} X_{i}$ is minimax.

  comment

• Paper 4, Section II, I

  Principles of Statistics

  Part II, 2009

  Consider the double dichotomy, where the loss is 0 for a correct decision and 1 for an incorrect decision. Describe the form of a Bayes decision rule. Assuming the equivalence of normal and extensive form analyses, deduce the Neyman-Pearson lemma.

  For a problem with random variable $X$ and real parameter $\theta$, define monotone likelihood ratio (MLR) and monotone test.

  Suppose the problem has MLR in a real statistic $T=t(X)$. Let $\phi$ be a monotone test, with power function $\gamma(\cdot)$, and let $\phi^{\prime}$ be any other test, with power function $\gamma^{\prime}(\cdot)$. Show that if $\theta_{1}>\theta_{0}$ and $\gamma\left(\theta_{0}\right)>\gamma^{\prime}\left(\theta_{0}\right)$, then $\gamma\left(\theta_{1}\right)>\gamma^{\prime}\left(\theta_{1}\right)$. Deduce that there exists $\theta^{*} \in[-\infty, \infty]$ such that $\gamma(\theta) \leqslant \gamma^{\prime}(\theta)$ for $\theta<\theta^{*}$, and $\gamma(\theta) \geqslant \gamma^{\prime}(\theta)$ for $\theta>\theta^{*}$.

  For an arbitrary prior distribution $\Pi$ with density $\pi(\cdot)$, and an arbitrary value $\theta^{*}$, show that the posterior odds

  $$\frac{\Pi\left(\theta>\theta^{*} \mid X=x\right)}{\Pi\left(\theta \leqslant \theta^{*} \mid X=x\right)}$$

  is a non-decreasing function of $t(x)$.

  comment

• Paper 1, Section II, I

  Principles of Statistics

  Part II, 2009

  (i) Let $X_{1}, \ldots, X_{n}$ be independent and identically distributed random variables, having the exponential distribution $\mathcal{E}(\lambda)$ with density $p(x \mid \lambda)=\lambda \exp (-\lambda x)$ for $x, \lambda>0$. Show that $T_{n}=\sum_{i=1}^{n} X_{i}$ is minimal sufficient and complete for $\lambda$.

  [You may assume uniqueness of Laplace transforms.]

  (ii) For given $x>0$, it is desired to estimate the quantity $\phi=\operatorname{Prob}\left(X_{1}>x \mid \lambda\right)$. Compute the Fisher information for $\phi$.

  (iii) State the Lehmann-Scheffé theorem. Show that the estimator $\tilde{\phi}_{n}$ of $\phi$ defined by

  $$\tilde{\phi}_{n}= \begin{cases}0, & \text { if } T_{n}<x, \\ \left(1-\frac{x}{T_{n}}\right)^{n-1}, & \text { if } T_{n} \geqslant x\end{cases}$$

  is the minimum variance unbiased estimator of $\phi$ based on $\left(X_{1}, \ldots, X_{n}\right)$. Without doing any computations, state whether or not the variance of $\tilde{\phi}_{n}$ achieves the Cramér-Rao lower bound, justifying your answer briefly.

  Let $k \leqslant n$. Show that $\mathbb{E}\left(\tilde{\phi}_{k} \mid T_{n}, \lambda\right)=\tilde{\phi}_{n}$.

  comment

• Paper 2, Section II, I

  Principles of Statistics

  Part II, 2009

  Suppose that the random vector $\mathbf{X}=\left(X_{1}, \ldots, X_{n}\right)$ has a distribution over $\mathbb{R}^{n}$ depending on a real parameter $\theta$, with everywhere positive density function $p(\mathbf{x} \mid \theta)$. Define the maximum likelihood estimator $\hat{\theta}$, the score variable $U$, the observed information $\hat{j}$ and the expected (Fisher) information $I$ for the problem of estimating $\theta$ from $\mathbf{X}$.

  For the case where the $\left(X_{i}\right)$ are independent and identically distributed, show that, as $n \rightarrow \infty, I^{-1 / 2} U \stackrel{d}{\rightarrow} \mathcal{N}(0,1)$. [You may assume sufficient conditions to allow interchange of integration over the sample space and differentiation with respect to the parameter.] State the asymptotic distribution of $\hat{\theta}$.

  The random vector $\mathbf{X}=\left(X_{1}, \ldots, X_{n}\right)$ is generated according to the rule

  $$X_{i+1}=\theta X_{i}+E_{i}$$

  where $X_{0}=1$ and the $\left(E_{i}\right)$ are independent and identically distributed from the standard normal distribution $\mathcal{N}(0,1)$. Write down the likelihood function for $\theta$ based on data $\mathbf{x}=\left(x_{1}, \ldots, x_{n}\right)$, find $\hat{\theta}$ and $\hat{j}$ and show that the pair $(\hat{\theta}, \hat{j})$ forms a minimal sufficient statistic.

  A Bayesian uses the improper prior density $\pi(\theta) \propto 1$. Show that, in the posterior, $S(\theta-\hat{\theta})$ (where $S$ is a statistic that you should identify) has the same distribution as $E_{1}$.

  comment

• Paper 1, Section II, J

  Principles of Statistics

  Part II, 2010

  The distribution of a random variable $X$ is obtained from the binomial distribution $\mathcal{B}(n ; \Pi)$ by conditioning on $X>0$; here $\Pi \in(0,1)$ is an unknown probability parameter and $n$ is known. Show that the distributions of $X$ form an exponential family and identify the natural sufficient statistic $T$, natural parameter $\Phi$, and cumulant function $k(\phi)$. Using general properties of the cumulant function, compute the mean and variance of $X$ when $\Pi=\pi$. Write down an equation for the maximum likelihood estimate $\widehat{\Pi}$of $\Pi$ and explain why, when $\Pi=\pi$, the distribution of $\widehat{\Pi}$is approximately normal $\mathcal{N}(\pi, \pi(1-\pi) / n)$ for large $n$.

  Suppose we observe $X=1$. It is suggested that, since the condition $X>0$ is then automatically satisfied, general principles of inference require that the inference to be drawn should be the same as if the distribution of $X$ had been $\mathcal{B}(n ; \Pi)$ and we had observed $X=1$. Comment briefly on this suggestion.

  comment

• Paper 2, Section II, J

  Principles of Statistics

  Part II, 2010

  Define the Kolmogorov-Smirnov statistic for testing the null hypothesis that real random variables $X_{1}, \ldots, X_{n}$ are independently and identically distributed with specified continuous, strictly increasing distribution function $F$, and show that its null distribution does not depend on $F$.

  A composite hypothesis $H_{0}$ specifies that, when the unknown positive parameter $\Theta$ takes value $\theta$, the random variables $X_{1}, \ldots, X_{n}$ arise independently from the uniform distribution $\mathrm{U}[0, \theta]$. Letting $J:=\arg \max _{1 \leqslant i \leqslant n} X_{i}$, show that, under $H_{0}$, the statistic $\left(J, X_{J}\right)$ is sufficient for $\Theta$. Show further that, given $\left\{J=j, X_{j}=\xi\right\}$, the random variables $\left(X_{i}: i \neq j\right)$ are independent and have the $\mathrm{U}[0, \xi]$ distribution. How might you apply the Kolmogorov-Smirnov test to test the hypothesis $H_{0}$ ?

  comment

• Paper 3, Section II, $\mathbf{2 7 J}$

  Principles of Statistics

  Part II, 2010

  Define the normal and extensive form solutions of a Bayesian statistical decision problem involving parameter $\Theta$, random variable $X$, and loss function $L(\theta, a)$. How are they related? Let $R_{0}=R_{0}(\Pi)$ be the Bayes loss of the optimal act when $\Theta \sim \Pi$ and no data can be observed. Express the Bayes risk $R_{1}$ of the optimal statistical decision rule in terms of $R_{0}$ and the joint distribution of $(\Theta, X)$.

  The real parameter $\Theta$ has distribution $\Pi$, having probability density function $\pi(\cdot)$. Consider the problem of specifying a set $S \subseteq \mathbb{R}$ such that the loss when $\Theta=\theta$ is $L(\theta, S)=c|S|-\mathbf{1}_{S}(\theta)$, where $\mathbf{1}_{S}$ is the indicator function of $S$, where $c>0$, and where $|S|=\int_{S} d x$. Show that the "highest density" region $S^{*}:=\{\theta: \pi(\theta) \geqslant c\}$ supplies a Bayes act for this decision problem, and explain why $R_{0}(\Pi) \leqslant 0$.

  For the case $\Theta \sim \mathcal{N}\left(\mu, \sigma^{2}\right)$, find an expression for $R_{0}$ in terms of the standard normal distribution function $\Phi$.

  Suppose now that $c=0.5$, that $\Theta \sim \mathcal{N}(0,1)$ and that $X \mid \Theta \sim \mathcal{N}(\Theta, 1 / 9)$. Show that $R_{1}<R_{0}$.

  comment

• Paper 4, Section II, $\mathbf{2 7 J}$

  Principles of Statistics

  Part II, 2010

  Define completeness and bounded completeness of a statistic $T$ in a statistical experiment.

  Random variables $X_{1}, X_{2}, X_{3}$ are generated as $X_{i}=\Theta^{1 / 2} Z+(1-\Theta)^{1 / 2} Y_{i}$, where $Z, Y_{1}, Y_{2}, Y_{3}$ are independently standard normal $\mathcal{N}(0,1)$, and the parameter $\Theta$ takes values in $(0,1)$. What is the joint distribution of $\left(X_{1}, X_{2}, X_{3}\right)$ when $\Theta=\theta$ ? Write down its density function, and show that a minimal sufficient statistic for $\Theta$ based on $\left(X_{1}, X_{2}, X_{3}\right)$ is $T=\left(T_{1}, T_{2}\right):=\left(\sum_{i=1}^{3} X_{i}^{2},\left(\sum_{i=1}^{3} X_{i}\right)^{2}\right)$.

  [Hint: You may use that if $I$ is the $n \times n$ identity matrix and $J$ is the $n \times n$ matrix all of whose entries are 1 , then $a I+b J$ has determinant $a^{n-1}(a+n b)$, and inverse $c I+d J$ with $c=1 / a, d=-b /(a(a+n b))$.]

  What is $\mathbb{E}_{\theta}\left(T_{1}\right) ?$ Is $T$ complete for $\Theta ?$

  Let $S:=\operatorname{Prob}\left(X_{1}^{2} \leqslant 1 \mid T\right)$. Show that $\mathbb{E}_{\theta}(S)$ is a positive constant $c$ which does not depend on $\theta$, but that $S$ is not identically equal to $c$. Is $T$ boundedly complete for $\Theta$ ?

  comment

• Paper 1, Section II, K

  Principles of Statistics

  Part II, 2011

  Define admissible, Bayes, minimax decision rules.

  A random vector $X=\left(X_{1}, X_{2}, X_{3}\right)^{\mathrm{T}}$ has independent components, where $X_{i}$ has the normal distribution $\mathcal{N}\left(\theta_{i}, 1\right)$ when the parameter vector $\Theta$ takes the value $\theta=\left(\theta_{1}, \theta_{2}, \theta_{3}\right)^{\mathrm{T}}$. It is required to estimate $\Theta$ by a point $a \in \mathbb{R}^{3}$, with loss function $L(\theta, a)=\|a-\theta\|^{2}$. What is the risk function of the maximum-likelihood estimator $\widehat{\Theta}:=X ?$ Show that $\widehat{\Theta}$is dominated by the estimator $\widetilde{\Theta}:=\left(1-\|X\|^{-2}\right) X$.

  comment

• Paper 2, Section II, K

  Principles of Statistics

  Part II, 2011

  Random variables $X_{1}, \ldots, X_{n}$ are independent and identically distributed from the normal distribution with unknown mean $\mathrm{M}$ and unknown precision (inverse variance) $H$. Show that the likelihood function, for data $X_{1}=x_{1}, \ldots, X_{n}=x_{n}$, is

  $$L_{n}(\mu, h) \propto h^{n / 2} \exp \left(-\frac{1}{2} h\left\{n(\bar{x}-\mu)^{2}+S\right\}\right)$$

  where $\bar{x}:=n^{-1} \sum_{i} x_{i}$ and $S:=\sum_{i}\left(x_{i}-\bar{x}\right)^{2}$.

  A bivariate prior distribution for $(\mathrm{M}, H)$ is specified, in terms of hyperparameters $\left(\alpha_{0}, \beta_{0}, m_{0}, \lambda_{0}\right)$, as follows. The marginal distribution of $H$ is $\Gamma\left(\alpha_{0}, \beta_{0}\right)$, with density

  $$\pi(h) \propto h^{\alpha_{0}-1} e^{-\beta_{0} h} \quad(h>0),$$

  and the conditional distribution of $\mathrm{M}$, given $H=h$, is normal with mean $m_{0}$ and precision $\lambda_{0} h$.

  Show that the conditional prior distribution of $H$, given $M=\mu$, is

  $$H \mid \mathrm{M}=\mu \quad \sim\left(\alpha_{0}+\frac{1}{2}, \beta_{0}+\frac{1}{2} \lambda_{0}\left(\mu-m_{0}\right)^{2}\right)$$

  Show that the posterior joint distribution of $(\mathrm{M}, H)$, given $X_{1}=x_{1}, \ldots, X_{n}=x_{n}$, has the same form as the prior, with updated hyperparameters $\left(\alpha_{n}, \beta_{n}, m_{n}, \lambda_{n}\right)$ which you should express in terms of the prior hyperparameters and the data.

  [You may use the identity

  $$p(t-a)^{2}+q(t-b)^{2}=(t-\delta)^{2}+p q(a-b)^{2}$$

  where $p+q=1$ and $\delta=p a+q b$.]

  Explain how you could implement Gibbs sampling to generate a random sample from the posterior joint distribution.

  comment

• Paper 3, Section II, $27 \mathrm{~K}$

  Principles of Statistics

  Part II, 2011

  Random variables $X_{1}, X_{2}, \ldots$ are independent and identically distributed from the exponential distribution $\mathcal{E}(\theta)$, with density function

  $$p_{X}(x \mid \theta)=\theta e^{-\theta x} \quad(x>0),$$

  when the parameter $\Theta$ takes value $\theta>0$. The following experiment is performed. First $X_{1}$ is observed. Thereafter, if $X_{1}=x_{1}, \ldots, X_{i}=x_{i}$ have been observed $(i \geqslant 1)$, a coin having probability $\alpha\left(x_{i}\right)$ of landing heads is tossed, where $\alpha: \mathbb{R} \rightarrow(0,1)$ is a known function and the coin toss is independent of the $X$ 's and previous tosses. If it lands heads, no further observations are made; if tails, $X_{i+1}$ is observed.

  Let $N$ be the total number of $X$ 's observed, and $\mathbf{X}:=\left(X_{1}, \ldots, X_{N}\right)$. Write down the likelihood function for $\Theta$ based on data $\mathbf{X}=\left(x_{1}, \ldots, x_{n}\right)$, and identify a minimal sufficient statistic. What does the likelihood principle have to say about inference from this experiment?

  Now consider the experiment that only records $Y:=X_{N}$. Show that the density function of $Y$ has the form

  $$p_{Y}(y \mid \theta)=\exp \{a(y)-k(\theta)-\theta y\}$$

  Assuming the function $a(\cdot)$ is twice differentiable and that both $p_{Y}(y \mid \theta)$ and $\partial p_{Y}(y \mid \theta) / \partial y$ vanish at 0 and $\infty$, show that $a^{\prime}(Y)$ is an unbiased estimator of $\Theta$, and find its variance.

  Stating clearly any general results you use, deduce that

  $$-k^{\prime \prime}(\theta) \mathbb{E}_{\theta}\left\{a^{\prime \prime}(Y)\right\} \geqslant 1 .$$

  comment

• Paper 4, Section II, K

  Principles of Statistics

  Part II, 2011

  What does it mean to say that a $(1 \times p)$ random vector $\xi$ has a multivariate normal distribution?

  Suppose $\xi=(X, Y)$ has the bivariate normal distribution with mean vector $\mu=\left(\mu_{X}, \mu_{Y}\right)$, and dispersion matrix

  $$\Sigma=\left(\begin{array}{cc} \sigma_{X X} & \sigma_{X Y} \\ \sigma_{X Y} & \sigma_{Y Y} \end{array}\right)$$

  Show that, with $\beta:=\sigma_{X Y} / \sigma_{X X}, Y-\beta X$ is independent of $X$, and thus that the conditional distribution of $Y$ given $X$ is normal with mean $\mu_{Y}+\beta\left(X-\mu_{X}\right)$ and variance $\sigma_{Y Y \cdot X}:=\sigma_{Y Y}-\sigma_{X Y}^{2} / \sigma_{X X}$.

  For $i=1, \ldots, n, \xi_{i}=\left(X_{i}, Y_{i}\right)$ are independent and identically distributed with the above distribution, where all elements of $\mu$ and $\Sigma$ are unknown. Let

  $$S=\left(\begin{array}{cc} S_{X X} & S_{X Y} \\ S_{X Y} & S_{Y Y} \end{array}\right):=\sum_{i=1}^{n}\left(\xi_{i}-\bar{\xi}\right)^{\mathrm{T}}\left(\xi_{i}-\bar{\xi}\right)$$

  where $\bar{\xi}:=n^{-1} \sum_{i=1}^{n} \xi_{i}$.

  The sample correlation coefficient is $r:=S_{X Y} / \sqrt{S_{X X} S_{Y Y}}$. Show that the distribution of $r$ depends only on the population correlation coefficient $\rho:=\sigma_{X Y} / \sqrt{\sigma_{X X} \sigma_{Y Y}}$.

  Student's $t$-statistic (on $n-2$ degrees of freedom) for testing the null hypothesis $H_{0}: \beta=0$ is

  $$t:=\frac{\widehat{\beta}}{\sqrt{S_{Y Y \cdot X} /(n-2) S_{X X}}},$$

  where $\widehat{\beta}:=S_{X Y} / S_{X X}$ and $S_{Y Y \cdot X}:=S_{Y Y}-S_{X Y}^{2} / S_{X X}$. Its density when $H_{0}$ is true is

  $$p(t)=C\left(1+\frac{t^{2}}{n-2}\right)^{-\frac{1}{2}(n-1)}$$

  where $C$ is a constant that need not be specified.

  Express $t$ in terms of $r$, and hence derive the density of $r$ when $\rho=0$.

  How could you use the sample correlation $r$ to test the hypothesis $\rho=0$ ?

  comment

• Paper 4, Section II, K

  Principles of Statistics

  Part II, 2012

  For $i=1, \ldots, n$, the pairs $\left(X_{i}, Y_{i}\right)$ have independent bivariate normal distributions, with $\mathbb{E}\left(X_{i}\right)=\mu_{X}, \mathbb{E}\left(Y_{i}\right)=\mu_{Y}, \operatorname{var}\left(X_{i}\right)=\operatorname{var}\left(Y_{i}\right)=\phi$, and $\operatorname{corr}\left(X_{i}, Y_{i}\right)=\rho$. The means $\mu_{X}, \mu_{Y}$ are known; the parameters $\phi>0$ and $\rho \in(-1,1)$ are unknown.

  Show that the joint distribution of all the variables belongs to an exponential family, and identify the natural sufficient statistic, natural parameter, and mean-value parameter. Hence or otherwise, find the maximum likelihood estimator $\hat{\rho}$ of $\rho$.

  Let $U_{i}:=X_{i}+Y_{i}, V_{i}:=X_{i}-Y_{i}$. What is the joint distribution of $\left(U_{i}, V_{i}\right) ?$

  Show that the distribution of

  $$\frac{(1+\hat{\rho}) /(1-\hat{\rho})}{(1+\rho) /(1-\rho)}$$

  is $F_{n}^{n}$. Hence describe a $(1-\alpha)$-level confidence interval for $\rho$. Briefly explain what would change if $\mu_{X}$ and $\mu_{Y}$ were also unknown.

  [Recall that the distribution $F_{\nu_{2}}^{\nu_{1}}$ is that of $\left(W_{1} / \nu_{1}\right) /\left(W_{2} / \nu_{2}\right)$, where, independently for $j=1$ and $j=2, W_{j}$ has the chi-squared distribution with $\nu_{j}$ degrees of freedom.]

  comment

• Paper 3, Section II, $27 \mathrm{~K}$

  Principles of Statistics

  Part II, 2012

  The parameter vector is $\Theta \equiv\left(\Theta_{1}, \Theta_{2}, \Theta_{3}\right)$, with $\Theta_{i}>0, \Theta_{1}+\Theta_{2}+\Theta_{3}=1$. Given $\boldsymbol{\Theta}=\boldsymbol{\theta} \equiv\left(\theta_{1}, \theta_{2}, \theta_{3}\right)$, the integer random vector $\boldsymbol{X}=\left(X_{1}, X_{2}, X_{3}\right)$ has a trinomial distribution, with probability mass function

  $$p(\boldsymbol{x} \mid \boldsymbol{\theta})=\frac{n !}{x_{1} ! x_{2} ! x_{3} !} \theta_{1}^{x_{1}} \theta_{2}^{x_{2}} \theta_{3}^{x_{3}}, \quad\left(x_{i} \geqslant 0, \sum_{i=1}^{3} x_{i}=n\right)$$

  Compute the score vector for the parameter $\Theta^{*}:=\left(\Theta_{1}, \Theta_{2}\right)$, and, quoting any relevant general result, use this to determine $\mathbb{E}\left(X_{i}\right)(i=1,2,3)$.

  Considering (1) as an exponential family with mean-value parameter $\Theta^{*}$, what is the corresponding natural parameter $\boldsymbol{\Phi} \equiv\left(\Phi_{1}, \Phi_{2}\right)$ ?

  Compute the information matrix $I$ for $\Theta^{*}$, which has $(i, j)$-entry

  $$I_{i j}=-\mathbb{E}\left(\frac{\partial^{2} l}{\partial \theta_{i} \partial \theta_{j}}\right) \quad(i, j=1,2)$$

  where $l$ denotes the log-likelihood function, based on $\boldsymbol{X}$, expressed in terms of $\left(\theta_{1}, \theta_{2}\right)$.

  Show that the variance of $\log \left(X_{1} / X_{3}\right)$ is asymptotic to $n^{-1}\left(\theta_{1}^{-1}+\theta_{3}^{-1}\right)$ as $n \rightarrow \infty$. [Hint. The information matrix $I_{\Phi}$ for $\boldsymbol{\Phi}$ is $I^{-1}$ and the dispersion matrix of the maximum likelihood estimator $\widehat{\boldsymbol{\Phi}}$behaves, asymptotically (for $n \rightarrow \infty$ ) as $I_{\Phi}^{-1}$.]

  comment

• Paper 2, Section II, $28 \mathrm{~K}$

  Principles of Statistics

  Part II, 2012

  Carefully defining all italicised terms, show that, if a sufficiently general method of inference respects both the Weak Sufficiency Principle and the Conditionality Principle, then it respects the Likelihood Principle.

  The position $X_{t}$ of a particle at time $t>0$ has the Normal distribution $\mathcal{N}(0, \phi t)$, where $\phi$ is the value of an unknown parameter $\Phi$; and the time, $T_{x}$, at which the particle first reaches position $x \neq 0$ has probability density function

  $$p_{x}(t)=\frac{|x|}{\sqrt{2 \pi \phi t^{3}}} \exp \left(-\frac{x^{2}}{2 \phi t}\right) \quad(t>0) .$$

  Experimenter $E_{1}$ observes $X_{\tau}$, and experimenter $E_{2}$ observes $T_{\xi}$, where $\tau>0, \xi \neq 0$ are fixed in advance. It turns out that $T_{\xi}=\tau$. What does the Likelihood Principle say about the inferences about $\Phi$ to be made by the two experimenters?

  $E_{1}$ bases his inference about $\Phi$ on the distribution and observed value of $X_{\tau}^{2} / \tau$, while $E_{2}$ bases her inference on the distribution and observed value of $\xi^{2} / T_{\xi}$. Show that these choices respect the Likelihood Principle.

  comment

• Paper 1, Section II, $28 \mathrm{~K}$

  Principles of Statistics

  Part II, 2012

  Prove that, if $T$ is complete sufficient for $\Theta$, and $S$ is a function of $T$, then $S$ is the minimum variance unbiased estimator of $\mathbb{E}(S \mid \Theta)$.

  When the parameter $\Theta$ takes a value $\theta>0$, observables $\left(X_{1}, \ldots, X_{n}\right)$ arise independently from the exponential distribution $\mathcal{E}(\theta)$, having probability density function

  $$p(x \mid \theta)=\theta e^{-\theta x} \quad(x>0) .$$

  Show that the family of distributions

  $$\Theta \sim \operatorname{Gamma}(\alpha, \beta) \quad(\alpha>0, \beta>0),$$

  with probability density function

  $$\pi(\theta)=\frac{\beta^{\alpha}}{\Gamma(\alpha)} \theta^{\alpha-1} e^{-\beta \theta} \quad(\theta>0),$$

  is a conjugate family for Bayesian inference about $\Theta$ (where $\Gamma(\alpha)$ is the Gamma function).

  Show that the expectation of $\Lambda:=\log \Theta$, under prior distribution (1), is $\psi(\alpha)-\log \beta$, where $\psi(\alpha):=(\mathrm{d} / \mathrm{d} \alpha) \log \Gamma(\alpha)$. What is the prior variance of $\Lambda$ ? Deduce the posterior expectation and variance of $\Lambda$, given $\left(X_{1}, \ldots, X_{n}\right)$.

  Let $\tilde{\Lambda}$ denote the limiting form of the posterior expectation of $\Lambda$ as $\alpha, \beta \downarrow 0$. Show that $\tilde{\Lambda}$ is the minimum variance unbiased estimator of $\Lambda$. What is its variance?

  comment

• Paper 4, Section II, $27 \mathrm{~K}$

  Principles of Statistics

  Part II, 2013

  Assuming only the existence and properties of the univariate normal distribution, define $\mathcal{N}_{p}(\underline{\mu}, \Sigma)$, the multivariate normal distribution with mean (row-)vector $\underline{\mu}$ and dispersion matrix $\Sigma$; and $W_{p}(\nu ; \Sigma)$, the Wishart distribution on integer $\nu>1$ degrees of freedom and with scale parameter $\Sigma$. Show that, if $\underline{X} \sim \mathcal{N}_{p}(\underline{\mu}, \Sigma), S \sim W_{p}(\nu ; \Sigma)$, and $\underline{b}(1 \times q), A(p \times q)$ are fixed, then $\underline{b}+\underline{X} A \sim \mathcal{N}_{q}(\underline{b}+\underline{\mu} A, \Phi), A^{\mathrm{T}} S A \sim W_{p}(\nu ; \Phi)$, where $\Phi=A^{\mathrm{T}} \Sigma A$.

  The random $(n \times p)$ matrix $X$ has rows that are independently distributed as $\mathcal{N}_{p}(\underline{\mathrm{M}}, \Sigma)$, where both parameters $\underline{\mathrm{M}}$ and $\Sigma$ are unknown. Let $\underline{\bar{X}}:=n^{-1} \mathbf{1}^{\mathrm{T}} X$, where 1 is the $(n \times 1)$ vector of $1 \mathrm{~s}$; and $S^{c}:=X^{\mathrm{T}} \Pi X$, with $\Pi:=I_{n}-n^{-1} 11^{\mathrm{T}}$. State the joint distribution of $\bar{X}$ and $S^{c}$ given the parameters.

  Now suppose $n>p$ and $\Sigma$ is positive definite. Hotelling's $T^{2}$ is defined as

  $$T^{2}:=n(\underline{\bar{X}}-\underline{\mathrm{M}})\left(\bar{S}^{c}\right)^{-1}(\underline{\bar{X}}-\underline{\mathrm{M}})^{\mathrm{T}}$$

  where $\bar{S}^{c}:=S^{c} / \nu$ with $\nu:=(n-1)$. Show that, for any values of $\underline{\mathrm{M}}$ and $\Sigma$,

  $$\left(\frac{\nu-p+1}{\nu p}\right) T^{2} \sim F_{\nu-p+1}^{p},$$

  the $F$ distribution on $p$ and $\nu-p+1$ degrees of freedom.

  [You may assume that:

  1. If $S \sim W_{p}(\nu ; \Sigma)$ and $\mathbf{a}$ is a fixed $(p \times 1)$ vector, then

  $$\frac{\mathbf{a}^{\mathrm{T}} \Sigma^{-1} \mathbf{a}}{\mathbf{a}^{\mathrm{T}} S^{-1} \mathbf{a}} \sim \chi_{\nu-p+1}^{2}$$

  2. If $V \sim \chi_{p}^{2}, W \sim \chi_{\lambda}^{2}$ are independent, then

  $$\frac{V / p}{W / \lambda} \sim F_{\lambda}^{p} \text {. }$$

  comment

• Paper 3, Section II, K

  Principles of Statistics

  Part II, 2013

  What is meant by a convex decision problem? State and prove a theorem to the effect that, in a convex decision problem, there is no point in randomising. [You may use standard terms without defining them.]

  The sample space, parameter space and action space are each the two-point set $\{1,2\}$. The observable $X$ takes value 1 with probability $2 / 3$ when the parameter $\Theta=1$, and with probability $3 / 4$ when $\Theta=2$. The loss function $L(\theta, a)$ is 0 if $a=\theta$, otherwise 1 . Describe all the non-randomised decision rules, compute their risk functions, and plot these as points in the unit square. Identify an inadmissible non-randomised decision rule, and a decision rule that dominates it.

  Show that the minimax rule has risk function $(8 / 17,8 / 17)$, and is Bayes against a prior distribution that you should specify. What is its Bayes risk? Would a Bayesian with this prior distribution be bound to use the minimax rule?

  comment

• Paper 1, Section II, K

  Principles of Statistics

  Part II, 2013

  When the real parameter $\Theta$ takes value $\theta$, variables $X_{1}, X_{2}, \ldots$ arise independently from a distribution $P_{\theta}$ having density function $p_{\theta}(x)$ with respect to an underlying measure $\mu$. Define the score variable $U_{n}(\theta)$ and the information function $I_{n}(\theta)$ for estimation of $\Theta$ based on $\boldsymbol{X}^{n}:=\left(X_{1}, \ldots, X_{n}\right)$, and relate $I_{n}(\theta)$ to $i(\theta):=I_{1}(\theta)$.

  State and prove the Cramér-Rao inequality for the variance of an unbiased estimator of $\Theta$. Under what conditions does this inequality become an equality? What is the form of the estimator in this case? [You may assume $\mathbb{E}_{\theta}\left\{U_{n}(\theta)\right\}=0, \operatorname{var}_{\theta}\left\{U_{n}(\theta)\right\}=I_{n}(\theta)$, and any further required regularity conditions, without comment.]

  Let $\widehat{\Theta}_{n}$ be the maximum likelihood estimator of $\Theta$ based on $\boldsymbol{X}^{n}$. What is the asymptotic distribution of $n^{\frac{1}{2}}\left(\widehat{\Theta}_{n}-\Theta\right)$ when $\Theta=\theta$ ?

  Suppose that, for each $n, \widehat{\Theta}_{n}$ is unbiased for $\Theta$, and the variance of $n^{\frac{1}{2}}\left(\widehat{\Theta}_{n}-\Theta\right)$ is exactly equal to its asymptotic variance. By considering the estimator $\alpha \widehat{\Theta}_{k}+(1-\alpha) \widehat{\Theta}_{n}$, or otherwise, show that, for $k<n, \operatorname{cov}_{\theta}\left(\widehat{\Theta}_{k}, \widehat{\Theta}_{n}\right)=\operatorname{var}_{\theta}\left(\widehat{\Theta}_{n}\right)$.

  comment

• Paper 2, Section II, K

  Principles of Statistics

  Part II, 2013

  Describe the Weak Sufficiency Principle (WSP) and the Strong Sufficiency Principle (SSP). Show that Bayesian inference with a fixed prior distribution respects WSP.

  A parameter $\Phi$ has a prior distribution which is normal with mean 0 and precision (inverse variance) $h_{\Phi} .$ Given $\Phi=\phi$, further parameters $\Theta:=\left(\Theta{ }_{i}: i=1, \ldots, I\right)$ have independent normal distributions with mean $\phi$ and precision $h_{\Theta}$. Finally, given both $\Phi=\phi$ and $\boldsymbol{\Theta}=\boldsymbol{\theta}:=\left(\theta_{1}, \ldots, \theta_{I}\right)$, observables $\boldsymbol{X}:=\left(X_{i j}: i=1, \ldots, I ; j=1, \ldots, J\right)$ are independent, $X_{i j}$ being normal with mean $\theta_{i}$, and precision $h_{X}$. The precision parameters $\left(h_{\Phi}, h_{\Theta}, h_{X}\right)$ are all fixed and known. Let $\overline{\boldsymbol{X}}:=\left(\bar{X}_{1}, \ldots, \bar{X}_{I}\right)$, where $\bar{X}_{i}:=\sum_{j=1}^{J} X_{i j} / J$. Show, directly from the definition of sufficiency, that $\overline{\boldsymbol{X}}$ is sufficient for $(\Phi, \Theta)$. [You may assume without proof that, if $Y_{1}, \ldots, Y_{n}$ have independent normal distributions with the same variance, and $\bar{Y}:=n^{-1} \sum_{i=1}^{n} Y_{i}$, then the vector $\left(Y_{1}-\bar{Y}, \ldots, Y_{n}-\bar{Y}\right)$ is independent of $\bar{Y}$.]

  For data-values $\boldsymbol{x}:=\left(x_{i j}: i=1, \ldots, I ; j=1, \ldots, J\right)$, determine the joint distribution, $\Pi_{\phi}$ say, of $\Theta$, given $\boldsymbol{X}=\boldsymbol{x}$ and $\Phi=\phi .$ What is the distribution of $\Phi$, given $\boldsymbol{\Theta}=\boldsymbol{\theta}$ and $\boldsymbol{X}=\boldsymbol{x} ?$

  Using these results, describe clearly how Gibbs sampling combined with RaoBlackwellisation could be applied to estimate the posterior joint distribution of $\Theta$, given $\boldsymbol{X}=\boldsymbol{x}$.

  comment

• Paper 4, Section II, J

  Principles of Statistics

  Part II, 2014

  Suppose you have at hand a pseudo-random number generator that can simulate an i.i.d. sequence of uniform $U[0,1]$ distributed random variables $U_{1}^{*}, \ldots, U_{N}^{*}$ for any $N \in \mathbb{N}$. Construct an algorithm to simulate an i.i.d. sequence $X_{1}^{*}, \ldots, X_{N}^{*}$ of standard normal $N(0,1)$ random variables. [Should your algorithm depend on the inverse of any cumulative probability distribution function, you are required to provide an explicit expression for this inverse function.]

  Suppose as a matter of urgency you need to approximately evaluate the integral

  $$I=\frac{1}{\sqrt{2 \pi}} \int_{\mathbb{R}} \frac{1}{(\pi+|x|)^{1 / 4}} e^{-x^{2} / 2} d x$$

  Find an approximation $I_{N}$ of this integral that requires $N$ simulation steps from your pseudo-random number generator, and which has stochastic accuracy

  $$\operatorname{Pr}\left(\left|I_{N}-I\right|>N^{-1 / 4}\right) \leqslant N^{-1 / 2}$$

  where Pr denotes the joint law of the simulated random variables. Justify your answer.

  comment

• Paper 3, Section II, J

  Principles of Statistics

  Part II, 2014

  State and prove Wilks' theorem about testing the simple hypothesis $H_{0}: \theta=\theta_{0}$, against the alternative $H_{1}: \theta \in \Theta \backslash\left\{\theta_{0}\right\}$, in a one-dimensional regular parametric model $\{f(\cdot, \theta): \theta \in \Theta\}, \Theta \subseteq \mathbb{R}$. [You may use without proof the results from lectures on the consistency and asymptotic distribution of maximum likelihood estimators, as well as on uniform laws of large numbers. Necessary regularity conditions can be assumed without statement.]

  Find the maximum likelihood estimator $\hat{\theta}_{n}$ based on i.i.d. observations $X_{1}, \ldots, X_{n}$ in a $N(0, \theta)$-model, $\theta \in \Theta=(0, \infty)$. Deduce the limit distribution as $n \rightarrow \infty$ of the sequence of statistics

  $$-n\left(\log \left(\overline{X^{2}}\right)-\left(\overline{X^{2}}-1\right)\right)$$

  where $\overline{X^{2}}=(1 / n) \sum_{i=1}^{n} X_{i}^{2}$ and $X_{1}, \ldots, X_{n}$ are i.i.d. $N(0,1)$.

  comment

• Paper 2, Section II, J

  Principles of Statistics

  Part II, 2014

  In a general decision problem, define the concepts of a Bayes rule and of admissibility. Show that a unique Bayes rule is admissible.

  Consider i.i.d. observations $X_{1}, \ldots, X_{n}$ from a $\operatorname{Poisson}(\theta), \theta \in \Theta=(0, \infty)$, model. Can the maximum likelihood estimator $\hat{\theta}_{M L E}$ of $\theta$ be a Bayes rule for estimating $\theta$ in quadratic risk for any prior distribution on $\theta$ that has a continuous probability density on $(0, \infty) ?$ Justify your answer.

  Now model the $X_{i}$ as i.i.d. copies of $X \mid \theta \sim \operatorname{Poisson}(\theta)$, where $\theta$ is drawn from a prior that is a Gamma distribution with parameters $\alpha>0$ and $\lambda>0$ (given below). Show that the posterior distribution of $\theta \mid X_{1}, \ldots, X_{n}$ is a Gamma distribution and find its parameters. Find the Bayes rule $\hat{\theta}_{B A Y E S}$ for estimating $\theta$ in quadratic risk for this prior. [The Gamma probability density function with parameters $\alpha>0, \lambda>0$ is given by

  $$f(\theta)=\frac{\lambda^{\alpha} \theta^{\alpha-1} e^{-\lambda \theta}}{\Gamma(\alpha)}, \quad \theta>0$$

  where $\Gamma(\alpha)$ is the usual Gamma function.]

  Finally assume that the $X_{i}$ have actually been generated from a fixed Poisson $\left(\theta_{0}\right)$ distribution, where $\theta_{0}>0$. Show that $\sqrt{n}\left(\hat{\theta}_{B A Y E S}-\hat{\theta}_{M L E}\right)$ converges to zero in probability and deduce the asymptotic distribution of $\sqrt{n}\left(\hat{\theta}_{B A Y E S}-\theta_{0}\right)$ under the joint law $P_{\theta_{0}}^{\mathbb{N}}$ of the random variables $X_{1}, X_{2}, \ldots$. [You may use standard results from lectures without proof provided they are clearly stated.]

  comment

• Paper 1, Section II, J

  Principles of Statistics

  Part II, 2014

  State without proof the inequality known as the Cramér-Rao lower bound in a parametric model $\{f(\cdot, \theta): \theta \in \Theta\}, \Theta \subseteq \mathbb{R}$. Give an example of a maximum likelihood estimator that attains this lower bound, and justify your answer.

  Give an example of a parametric model where the maximum likelihood estimator based on observations $X_{1}, \ldots, X_{n}$ is biased. State without proof an analogue of the Cramér-Rao inequality for biased estimators.

  Define the concept of a minimax decision rule, and show that the maximum likelihood estimator $\hat{\theta}_{M L E}$ based on $X_{1}, \ldots, X_{n}$ in a $N(\theta, 1)$ model is minimax for estimating $\theta \in \Theta=\mathbb{R}$ in quadratic risk.

  comment

• Paper 4, Section II, $\mathbf{2 4 J}$

  Principles of Statistics

  Part II, 2015

  Given independent and identically distributed observations $X_{1}, \ldots, X_{n}$ with finite mean $E\left(X_{1}\right)=\mu$ and variance $\operatorname{Var}\left(X_{1}\right)=\sigma^{2}$, explain the notion of a bootstrap sample $X_{1}^{b}, \ldots, X_{n}^{b}$, and discuss how you can use it to construct a confidence interval $C_{n}$ for $\mu$.

  Suppose you can operate a random number generator that can simulate independent uniform random variables $U_{1}, \ldots, U_{n}$ on $[0,1]$. How can you use such a random number generator to simulate a bootstrap sample?

  Suppose that $\left(F_{n}: n \in \mathbb{N}\right)$ and $F$ are cumulative probability distribution functions defined on the real line, that $F_{n}(t) \rightarrow F(t)$ as $n \rightarrow \infty$ for every $t \in \mathbb{R}$, and that $F$ is continuous on $\mathbb{R}$. Show that, as $n \rightarrow \infty$,

  $$\sup _{t \in \mathbb{R}}\left|F_{n}(t)-F(t)\right| \rightarrow 0 .$$

  State (without proof) the theorem about the consistency of the bootstrap of the mean, and use it to give an asymptotic justification of the confidence interval $C_{n}$. That is, prove that as $n \rightarrow \infty, P^{\mathbb{N}}\left(\mu \in C_{n}\right) \rightarrow 1-\alpha$ where $P^{\mathbb{N}}$ is the joint distribution of $X_{1}, X_{2}, \ldots$

  [You may use standard facts of stochastic convergence and the Central Limit Theorem without proof.]

  comment

• Paper 3, Section II, J

  Principles of Statistics

  Part II, 2015

  Define what it means for an estimator $\hat{\theta}$ of an unknown parameter $\theta$ to be consistent.

  Let $S_{n}$ be a sequence of random real-valued continuous functions defined on $\mathbb{R}$ such that, as $n \rightarrow \infty, S_{n}(\theta)$ converges to $S(\theta)$ in probability for every $\theta \in \mathbb{R}$, where $S: \mathbb{R} \rightarrow \mathbb{R}$ is non-random. Suppose that for some $\theta_{0} \in \mathbb{R}$ and every $\varepsilon>0$ we have

  $$S\left(\theta_{0}-\varepsilon\right)<0<S\left(\theta_{0}+\varepsilon\right),$$

  and that $S_{n}$ has exactly one zero $\hat{\theta}_{n}$ for every $n \in \mathbb{N}$. Show that $\hat{\theta}_{n} \rightarrow \theta_{0}$ as $n \rightarrow \infty$, and deduce from this that the maximum likelihood estimator (MLE) based on observations $X_{1}, \ldots, X_{n}$ from a $N(\theta, 1), \theta \in \mathbb{R}$ model is consistent.

  Now consider independent observations $\mathbf{X}_{1}, \ldots, \mathbf{X}_{n}$ of bivariate normal random vectors

  $$\mathbf{X}_{i}=\left(X_{1 i}, X_{2 i}\right)^{T} \sim N_{2}\left[\left(\mu_{i}, \mu_{i}\right)^{T}, \sigma^{2} I_{2}\right], \quad i=1, \ldots, n,$$

  where $\mu_{i} \in \mathbb{R}, \sigma>0$ and $I_{2}$ is the $2 \times 2$ identity matrix. Find the MLE $\hat{\mu}=\left(\hat{\mu}_{1}, \ldots, \hat{\mu}_{n}\right)^{T}$ of $\mu=\left(\mu_{1}, \ldots, \mu_{n}\right)^{T}$ and show that the MLE of $\sigma^{2}$ equals

  $$\hat{\sigma}^{2}=\frac{1}{n} \sum_{i=1}^{n} s_{i}^{2}, \quad s_{i}^{2}=\frac{1}{2}\left[\left(X_{1 i}-\hat{\mu}_{i}\right)^{2}+\left(X_{2 i}-\hat{\mu}_{i}\right)^{2}\right]$$

  Show that $\hat{\sigma}^{2}$ is not consistent for estimating $\sigma^{2}$. Explain briefly why the MLE fails in this model.

  [You may use the Law of Large Numbers without proof.]

  comment

• Paper 2, Section II, J

  Principles of Statistics

  Part II, 2015

  Consider a random variable $X$ arising from the binomial distribution $\operatorname{Bin}(n, \theta)$, $\theta \in \Theta=[0,1]$. Find the maximum likelihood estimator $\hat{\theta}_{M L E}$ and the Fisher information $I(\theta)$ for $\theta \in \Theta$.

  Now consider the following priors on $\Theta$ :

  (i) a uniform $U([0,1])$ prior on $[0,1]$,

  (ii) a prior with density $\pi(\theta)$ proportional to $\sqrt{I(\theta)}$,

  (iii) a $\operatorname{Beta}(\sqrt{n} / 2, \sqrt{n} / 2)$ prior.

  Find the means $E[\theta \mid X]$ and modes $m_{\theta} \mid X$ of the posterior distributions corresponding to the prior distributions (i)-(iii). Which of these posterior decision rules coincide with $\hat{\theta}_{M L E}$ ? Which one is minimax for quadratic risk? Justify your answers.

  [You may use the following properties of the $\operatorname{Beta}(a, b)(a>0, b>0)$ distribution. Its density $f(x ; a, b), x \in[0,1]$, is proportional to $x^{a-1}(1-x)^{b-1}$, its mean is equal to $a /(a+b)$, and its mode is equal to

  $$\frac{\max (a-1,0)}{\max (a, 1)+\max (b, 1)-2}$$

  provided either $a>1$ or $b>1$.

  You may further use the fact that a unique Bayes rule of constant risk is a unique minimax rule for that risk.]

  comment

• Paper 1, Section II, J

  Principles of Statistics

  Part II, 2015

  Consider a normally distributed random vector $X \in \mathbb{R}^{p}$ modelled as $X \sim N\left(\theta, I_{p}\right)$ where $\theta \in \mathbb{R}^{p}, I_{p}$ is the $p \times p$ identity matrix, and where $p \geqslant 3$. Define the Stein estimator $\hat{\theta}_{S T E I N}$ of $\theta$.

  Prove that $\hat{\theta}_{S T E I N}$ dominates the estimator $\tilde{\theta}=X$ for the risk function induced by quadratic loss

  $$\ell(a, \theta)=\sum_{i=1}^{p}\left(a_{i}-\theta_{i}\right)^{2}, \quad a \in \mathbb{R}^{p}$$

  Show however that the worst case risks coincide, that is, show that

  $$\sup _{\theta \in \mathbb{R}^{p}} E_{\theta} \ell(X, \theta)=\sup _{\theta \in \mathbb{R}^{p}} E_{\theta} \ell\left(\hat{\theta}_{S T E I N}, \theta\right)$$

  [You may use Stein's lemma without proof, provided it is clearly stated.]

  comment

• Paper 3, Section II, J

  Principles of Statistics

  Part II, 2016

  Let $X_{1}, \ldots, X_{n}$ be i.i.d. random variables from a $N(\theta, 1)$ distribution, $\theta \in \mathbb{R}$, and consider a Bayesian model $\theta \sim N\left(0, v^{2}\right)$ for the unknown parameter, where $v>0$ is a fixed constant.

  (a) Derive the posterior distribution $\Pi\left(\cdot \mid X_{1}, \ldots, X_{n}\right)$ of $\theta \mid X_{1}, \ldots, X_{n}$.

  (b) Construct a credible set $C_{n} \subset \mathbb{R}$ such that

  (i) $\Pi\left(C_{n} \mid X_{1}, \ldots, X_{n}\right)=0.95$ for every $n \in \mathbb{N}$, and

  (ii) for any $\theta_{0} \in \mathbb{R}$,

  $$P_{\theta_{0}}^{\mathbb{N}}\left(\theta_{0} \in C_{n}\right) \rightarrow 0.95 \quad \text { as } n \rightarrow \infty,$$

  where $P_{\theta}^{\mathbb{N}}$ denotes the distribution of the infinite sequence $X_{1}, X_{2}, \ldots$ when drawn independently from a fixed $N(\theta, 1)$ distribution.

  [You may use the central limit theorem.]

  comment

• Paper 2, Section II,

  Principles of Statistics

  Part II, 2016

  (a) State and prove the Cramér-Rao inequality in a parametric model $\{f(\theta): \theta \in \Theta\}$, where $\Theta \subseteq \mathbb{R}$. [Necessary regularity conditions on the model need not be specified.]

  (b) Let $X_{1}, \ldots, X_{n}$ be i.i.d. Poisson random variables with unknown parameter $E X_{1}=\theta>0$. For $\bar{X}_{n}=(1 / n) \sum_{i=1}^{n} X_{i}$ and $S^{2}=(n-1)^{-1} \sum_{i=1}^{n}\left(X_{i}-\bar{X}_{n}\right)^{2}$ define

  $$T_{\alpha}=\alpha \bar{X}_{n}+(1-\alpha) S^{2}, \quad 0 \leqslant \alpha \leqslant 1$$

  Show that $\operatorname{Var}_{\theta}\left(T_{\alpha}\right) \geqslant \operatorname{Var}_{\theta}\left(\bar{X}_{n}\right)$ for all values of $\alpha, \theta$.

  Now suppose $\tilde{\theta}=\tilde{\theta}\left(X_{1}, \ldots, X_{n}\right)$ is an estimator of $\theta$ with possibly nonzero bias $B(\theta)=E_{\theta} \tilde{\theta}-\theta$. Suppose the function $B$ is monotone increasing on $(0, \infty)$. Prove that the mean-squared errors satisfy

  $$E_{\theta}\left(\tilde{\theta}_{n}-\theta\right)^{2} \geqslant E_{\theta}\left(\bar{X}_{n}-\theta\right)^{2} \text { for all } \theta \in \Theta$$

  comment

• Paper 4, Section II, J

  Principles of Statistics

  Part II, 2016

  Consider a decision problem with parameter space $\Theta$. Define the concepts of a Bayes decision rule $\delta_{\pi}$ and of a least favourable prior.

  Suppose $\pi$ is a prior distribution on $\Theta$ such that the Bayes risk of the Bayes rule equals $\sup _{\theta \in \Theta} R\left(\delta_{\pi}, \theta\right)$, where $R(\delta, \theta)$ is the risk function associated to the decision problem. Prove that $\delta_{\pi}$ is least favourable.

  Now consider a random variable $X$ arising from the binomial distribution $\operatorname{Bin}(n, \theta)$, where $\theta \in \Theta=[0,1]$. Construct a least favourable prior for the squared risk $R(\delta, \theta)=E_{\theta}(\delta(X)-\theta)^{2}$. [You may use without proof the fact that the Bayes rule for quadratic risk is given by the posterior mean.]

  comment

• Paper 1, Section II, $\mathbf{2 7 J}$

  Principles of Statistics

  Part II, 2016

  Derive the maximum likelihood estimator $\hat{\theta}_{n}$ based on independent observations $X_{1}, \ldots, X_{n}$ that are identically distributed as $N(\theta, 1)$, where the unknown parameter $\theta$ lies in the parameter space $\Theta=\mathbb{R}$. Find the limiting distribution of $\sqrt{n}\left(\widehat{\theta}_{n}-\theta\right)$ as $n \rightarrow \infty$.

  Now define

  $$\begin{array}{rll} \tilde{\theta}_{n} & =\widehat{\theta}_{n} & \text { whenever }\left|\widehat{\theta}_{n}\right|>n^{-1 / 4}, \\ & =0 & \text { otherwise, } \end{array}$$

  $$\begin{aligned} & =0 \text { otherwise, } \end{aligned}$$

  and find the limiting distribution of $\sqrt{n}\left(\tilde{\theta}_{n}-\theta\right)$ as $n \rightarrow \infty$.

  Calculate

  $$\lim _{n \rightarrow \infty} \sup _{\theta \in \Theta} n E_{\theta}\left(T_{n}-\theta\right)^{2}$$

  for the choices $T_{n}=\widehat{\theta}_{n}$ and $T_{n}=\widetilde{\theta}_{n}$. Based on the above findings, which estimator $T_{n}$ of $\theta$ would you prefer? Explain your answer.

  [Throughout, you may use standard facts of stochastic convergence, such as the central limit theorem, provided they are clearly stated.]

  comment

• Paper 2, Section II, $26 \mathrm{~K}$

  Principles of Statistics

  Part II, 2017

  We consider the problem of estimating $\theta$ in the model $\{f(x, \theta): \theta \in(0, \infty)\}$, where

  $$f(x, \theta)=(1-\alpha)(x-\theta)^{-\alpha} 1\{x \in[\theta, \theta+1]\}$$

  Here $1\{A\}$ is the indicator of the set $A$, and $\alpha \in(0,1)$ is known. This estimation is based on a sample of $n$ i.i.d. $X_{1}, \ldots, X_{n}$, and we denote by $X_{(1)}<\ldots<X_{(n)}$ the ordered sample.

  (a) Compute the mean and the variance of $X_{1}$. Construct an unbiased estimator of $\theta$ taking the form $\tilde{\theta}_{n}=\bar{X}_{n}+c(\alpha)$, where $\bar{X}_{n}=n^{-1} \sum_{i=1}^{n} X_{i}$, specifying $c(\alpha)$.

  (b) Show that $\tilde{\theta}_{n}$ is consistent and find the limit in distribution of $\sqrt{n}\left(\tilde{\theta}_{n}-\theta\right)$. Justify your answer, citing theorems that you use.

  (c) Find the maximum likelihood estimator $\hat{\theta}_{n}$ of $\theta$. Compute $\mathbf{P}\left(\hat{\theta}_{n}-\theta>t\right)$ for all real $t$. Is $\hat{\theta}_{n}$ unbiased?

  (d) For $t>0$, show that $\mathbf{P}\left(n^{\beta}\left(\hat{\theta}_{n}-\theta\right)>t\right)$ has a limit in $(0,1)$ for some $\beta>0$. Give explicitly the value of $\beta$ and the limit. Why should one favour using $\hat{\theta}_{n}$ over $\tilde{\theta}_{n}$ ?

  comment

• Paper 3, Section II, $26 \mathrm{~K}$

  Principles of Statistics

  Part II, 2017

  We consider the problem of estimating an unknown $\theta_{0}$ in a statistical model $\{f(x, \theta), \theta \in \Theta\}$ where $\Theta \subset \mathbb{R}$, based on $n$ i.i.d. observations $X_{1}, \ldots, X_{n}$ whose distribution has p.d.f. $f\left(x, \theta_{0}\right)$.

  In all the parts below you may assume that the model satisfies necessary regularity conditions.

  (a) Define the score function $S_{n}$ of $\theta$. Prove that $S_{n}\left(\theta_{0}\right)$ has mean 0 .

  (b) Define the Fisher Information $I(\theta)$. Show that it can also be expressed as

  $$I(\theta)=-\mathbb{E}_{\theta}\left[\frac{d^{2}}{d \theta^{2}} \log f\left(X_{1}, \theta\right)\right]$$

  (c) Define the maximum likelihood estimator $\hat{\theta}_{n}$ of $\theta$. Give without proof the limits of $\hat{\theta}_{n}$ and of $\sqrt{n}\left(\hat{\theta}_{n}-\theta_{0}\right.$ ) (in a manner which you should specify). [Be as precise as possible when describing a distribution.]

  (d) Let $\psi: \Theta \rightarrow \mathbb{R}$ be a continuously differentiable function, and $\tilde{\theta}_{n}$ another estimator of $\theta_{0}$ such that $\left|\hat{\theta}_{n}-\tilde{\theta}_{n}\right| \leqslant 1 / n$ with probability 1 . Give the limits of $\psi\left(\tilde{\theta}_{n}\right)$ and of $\sqrt{n}\left(\psi\left(\tilde{\theta}_{n}\right)-\psi\left(\theta_{0}\right)\right)$ (in a manner which you should specify).

  comment

• Paper 4, Section II, $27 \mathrm{~K}$

  Principles of Statistics

  Part II, 2017

  For the statistical model $\left\{\mathcal{N}_{d}(\theta, \Sigma), \theta \in \mathbb{R}^{d}\right\}$, where $\Sigma$ is a known, positive-definite $d \times d$ matrix, we want to estimate $\theta$ based on $n$ i.i.d. observations $X_{1}, \ldots, X_{n}$ with distribution $\mathcal{N}_{d}(\theta, \Sigma)$.

  (a) Derive the maximum likelihood estimator $\hat{\theta}_{n}$ of $\theta$. What is the distribution of $\hat{\theta}_{n}$ ?

  (b) For $\alpha \in(0,1)$, construct a confidence region $C_{n}^{\alpha}$ such that $\mathbf{P}_{\theta}\left(\theta \in C_{n}^{\alpha}\right)=1-\alpha$.

  (c) For $\Sigma=I_{d}$, compute the maximum likelihood estimator of $\theta$ for the following parameter spaces:

  (i) $\Theta=\left\{\theta:\|\theta\|_{2}=1\right\}$.

  (ii) $\Theta=\left\{\theta: v^{\top} \theta=0\right\}$ for some unit vector $v \in \mathbb{R}^{d}$.

  (d) For $\Sigma=I_{d}$, we want to test the null hypothesis $\Theta_{0}=\{0\}$ (i.e. $\left.\theta=0\right)$ against the composite alternative $\Theta_{1}=\mathbb{R}^{d} \backslash\{0\}$. Compute the likelihood ratio statistic $\Lambda\left(\Theta_{1}, \Theta_{0}\right)$ and give its distribution under the null hypothesis. Compare this result with the statement of Wilks' theorem.

  comment

• Paper 1, Section II, $28 \mathrm{~K}$

  Principles of Statistics

  Part II, 2017

  For a positive integer $n$, we want to estimate the parameter $p$ in the binomial statistical model $\{\operatorname{Bin}(n, p), p \in[0,1]\}$, based on an observation $X \sim \operatorname{Bin}(n, p)$.

  (a) Compute the maximum likelihood estimator for $p$. Show that the posterior distribution for $p$ under a uniform prior on $[0,1]$ is $\operatorname{Beta}(a, b)$, and specify $a$ and $b$. [The p.d.f. of $\operatorname{Beta}(a, b)$ is given by

  $$\left.f_{a, b}(p)=\frac{(a+b-1) !}{(a-1) !(b-1) !} p^{a-1}(1-p)^{b-1} .\right]$$

  (b) (i) For a risk function $L$, define the risk of an estimator $\hat{p}$ of $p$, and the Bayes risk under a prior $\pi$ for $p$.

  (ii) Under the loss function

  $$L(\hat{p}, p)=\frac{(\hat{p}-p)^{2}}{p(1-p)}$$

  find a Bayes optimal estimator for the uniform prior. Give its risk as a function of $p$.

  (iii) Give a minimax optimal estimator for the loss function $L$ given above. Justify your answer.

  comment

• Paper 4, Section II, $28 \mathrm{~K}$

  Principles of Statistics

  Part II, 2018

  Let $g: \mathbb{R} \rightarrow \mathbb{R}$ be an unknown function, twice continuously differentiable with $\left|g^{\prime \prime}(x)\right| \leqslant M$ for all $x \in \mathbb{R}$. For some $x_{0} \in \mathbb{R}$, we know the value $g\left(x_{0}\right)$ and we wish to estimate its derivative $g^{\prime}\left(x_{0}\right)$. To do so, we have access to a pseudo-random number generator that gives $U_{1}^{*}, \ldots, U_{N}^{*}$ i.i.d. uniform over $[0,1]$, and a machine that takes input $x_{1}, \ldots, x_{N} \in \mathbb{R}$ and returns $g\left(x_{i}\right)+\varepsilon_{i}$, where the $\varepsilon_{i}$ are i.i.d. $\mathcal{N}\left(0, \sigma^{2}\right)$.

  (a) Explain how this setup allows us to generate $N$ independent $X_{i}=x_{0}+h Z_{i}$, where the $Z_{i}$ take value 1 or $-1$ with probability $1 / 2$, for any $h>0$.

  (b) We denote by $Y_{i}$ the output $g\left(X_{i}\right)+\varepsilon_{i}$. Show that for some independent $\xi_{i} \in \mathbb{R}$

  $$Y_{i}-g\left(x_{0}\right)=h Z_{i} g^{\prime}\left(x_{0}\right)+\frac{h^{2}}{2} g^{\prime \prime}\left(\xi_{i}\right)+\varepsilon_{i}$$

  (c) Using the intuition given by the least-squares estimator, justify the use of the estimator $\hat{g}_{N}$ given by

  $$\hat{g}_{N}=\frac{1}{N} \sum_{i=1}^{N} \frac{Z_{i}\left(Y_{i}-g\left(x_{0}\right)\right)}{h}$$

  (d) Show that

  $$\mathbb{E}\left[\left|\hat{g}_{N}-g^{\prime}\left(x_{0}\right)\right|^{2}\right] \leqslant \frac{h^{2} M^{2}}{4}+\frac{\sigma^{2}}{N h^{2}} .$$

  Show that for some choice $h_{N}$ of parameter $h$, this implies

  $$\mathbb{E}\left[\left|\hat{g}_{N}-g^{\prime}\left(x_{0}\right)\right|^{2}\right] \leqslant \frac{\sigma M}{\sqrt{N}}$$

  comment

• Paper 3, Section II, K

  Principles of Statistics

  Part II, 2018

  In the model $\left\{\mathcal{N}\left(\theta, I_{p}\right), \theta \in \mathbb{R}^{p}\right\}$ of a Gaussian distribution in dimension $p$, with unknown mean $\theta$ and known identity covariance matrix $I_{p}$, we estimate $\theta$ based on a sample of i.i.d. observations $X_{1}, \ldots, X_{n}$ drawn from $\mathcal{N}\left(\theta_{0}, I_{p}\right)$.

  (a) Define the Fisher information $I\left(\theta_{0}\right)$, and compute it in this model.

  (b) We recall that the observed Fisher information $i_{n}(\theta)$ is given by

  $$i_{n}(\theta)=\frac{1}{n} \sum_{i=1}^{n} \nabla_{\theta} \log f\left(X_{i}, \theta\right) \nabla_{\theta} \log f\left(X_{i}, \theta\right)^{\top}$$

  Find the limit of $\hat{i}_{n}=i_{n}\left(\hat{\theta}_{M L E}\right)$, where $\hat{\theta}_{M L E}$ is the maximum likelihood estimator of $\theta$ in this model.

  (c) Define the Wald statistic $W_{n}(\theta)$ and compute it. Give the limiting distribution of $W_{n}\left(\theta_{0}\right)$ and explain how it can be used to design a confidence interval for $\theta_{0}$.

  [You may use results from the course provided that you state them clearly.]

  comment

• Paper 2, Section II, $28 K$

  Principles of Statistics

  Part II, 2018

  We consider the model $\left\{\mathcal{N}\left(\theta, I_{p}\right), \theta \in \mathbb{R}^{p}\right\}$ of a Gaussian distribution in dimension $p \geqslant 3$, with unknown mean $\theta$ and known identity covariance matrix $I_{p}$. We estimate $\theta$ based on one observation $X \sim \mathcal{N}\left(\theta, I_{p}\right)$, under the loss function

  $$\ell(\theta, \delta)=\|\theta-\delta\|_{2}^{2}$$

  (a) Define the risk of an estimator $\hat{\theta}$. Compute the maximum likelihood estimator $\hat{\theta}_{M L E}$ of $\theta$ and its risk for any $\theta \in \mathbb{R}^{p}$.

  (b) Define what an admissible estimator is. Is $\hat{\theta}_{M L E}$ admissible?

  (c) For any $c>0$, let $\pi_{c}(\theta)$ be the prior $\mathcal{N}\left(0, c^{2} I_{p}\right)$. Find a Bayes optimal estimator $\hat{\theta}_{c}$ under this prior with the quadratic loss, and compute its Bayes risk.

  (d) Show that $\hat{\theta}_{M L E}$ is minimax.

  [You may use results from the course provided that you state them clearly.]

  comment

• Paper 1, Section II, $29 \mathrm{~K}$

  Principles of Statistics

  Part II, 2018

  A scientist wishes to estimate the proportion $\theta \in(0,1)$ of presence of a gene in a population of flies of size $n$. Every fly receives a chromosome from each of its two parents, each carrying the gene $A$ with probability $(1-\theta)$ or the gene $B$ with probability $\theta$, independently. The scientist can observe if each fly has two copies of the gene A (denoted by AA), two copies of the gene $B$ (denoted by BB) or one of each (denoted by AB). We let $n_{\mathrm{AA}}, n_{\mathrm{BB}}$, and $n_{\mathrm{AB}}$ denote the number of each observation among the $n$ flies.

  (a) Give the probability of each observation as a function of $\theta$, denoted by $f(X, \theta)$, for all three values $X=\mathrm{AA}, \mathrm{BB}$, or $\mathrm{AB}$.

  (b) For a vector $w=\left(w_{\mathrm{AA}}, w_{\mathrm{BB}}, w_{\mathrm{AB}}\right)$, we let $\hat{\theta}_{w}$ denote the estimator defined by

  $$\hat{\theta}_{w}=w_{\mathrm{AA}} \frac{n_{\mathrm{AA}}}{n}+w_{\mathrm{BB}} \frac{n_{\mathrm{BB}}}{n}+w_{\mathrm{AB}} \frac{n_{\mathrm{AB}}}{n} .$$

  Find the unique vector $w^{*}$ such that $\hat{\theta}_{w^{*}}$ is unbiased. Show that $\hat{\theta}_{w^{*}}$ is a consistent estimator of $\theta$.

  (c) Compute the maximum likelihood estimator of $\theta$ in this model, denoted by $\hat{\theta}_{M L E}$. Find the limiting distribution of $\sqrt{n}\left(\hat{\theta}_{M L E}-\theta\right)$. [You may use results from the course, provided that you state them clearly.]

  comment

• Paper 4, Section II, J

  Principles of Statistics

  Part II, 2019

  We consider a statistical model $\{f(\cdot, \theta): \theta \in \Theta\}$.

  (a) Define the maximum likelihood estimator (MLE) and the Fisher information $I(\theta) .$

  (b) Let $\Theta=\mathbb{R}$ and assume there exist a continuous one-to-one function $\mu: \mathbb{R} \rightarrow \mathbb{R}$ and a real-valued function $h$ such that

  $$\mathbb{E}_{\theta}[h(X)]=\mu(\theta) \quad \forall \theta \in \mathbb{R}$$

  (i) For $X_{1}, \ldots, X_{n}$ i.i.d. from the model for some $\theta_{0} \in \mathbb{R}$, give the limit in almost sure sense of

  $$\hat{\mu}_{n}=\frac{1}{n} \sum_{i=1}^{n} h\left(X_{i}\right)$$

  Give a consistent estimator $\hat{\theta}_{n}$ of $\theta_{0}$ in terms of $\hat{\mu}_{n}$.

  (ii) Assume further that $\mathbb{E}_{\theta_{0}}\left[h(X)^{2}\right]<\infty$ and that $\mu$ is continuously differentiable and strictly monotone. What is the limit in distribution of $\sqrt{n}\left(\hat{\theta}_{n}-\theta_{0}\right)$. Assume too that the statistical model satisfies the usual regularity assumptions. Do you necessarily expect $\operatorname{Var}\left(\hat{\theta}_{n}\right) \geqslant\left(n I\left(\theta_{0}\right)\right)^{-1}$ for all $n$ ? Why?

  (iii) Propose an alternative estimator for $\theta_{0}$ with smaller bias than $\hat{\theta}_{n}$ if $B_{n}\left(\theta_{0}\right)=$ $\mathbb{E}_{\theta_{0}}\left[\hat{\theta}_{n}\right]-\theta_{0}=\frac{a}{n}+\frac{b}{n^{2}}+O\left(\frac{1}{n^{3}}\right)$ for some $a, b \in \mathbb{R}$ with $a \neq 0$.

  (iv) Further to all the assumptions in iii), assume that the MLE for $\theta_{0}$ is of the form

  $$\hat{\theta}_{M L E}=\frac{1}{n} \sum_{i=1}^{n} h\left(X_{i}\right)$$

  What is the link between the Fisher information at $\theta_{0}$ and the variance of $h(X)$ ? What does this mean in terms of the precision of the estimator and why?

  [You may use results from the course, provided you state them clearly.]

  comment

• Paper 3, Section II, J

  Principles of Statistics

  Part II, 2019

  We consider the exponential model $\{f(\cdot, \theta): \theta \in(0, \infty)\}$, where

  $$f(x, \theta)=\theta e^{-\theta x} \quad \text { for } x \geqslant 0$$

  We observe an i.i.d. sample $X_{1}, \ldots, X_{n}$ from the model.

  (a) Compute the maximum likelihood estimator $\hat{\theta}_{M L E}$ for $\theta$. What is the limit in distribution of $\sqrt{n}\left(\hat{\theta}_{M L E}-\theta\right)$ ?

  (b) Consider the Bayesian setting and place a $\operatorname{Gamma}(\alpha, \beta), \alpha, \beta>0$, prior for $\theta$ with density

  $$\pi(\theta)=\frac{\beta^{\alpha}}{\Gamma(\alpha)} \theta^{\alpha-1} \exp (-\beta \theta) \quad \text { for } \theta>0$$

  where $\Gamma$ is the Gamma function satisfying $\Gamma(\alpha+1)=\alpha \Gamma(\alpha)$ for all $\alpha>0$. What is the posterior distribution for $\theta$ ? What is the Bayes estimator $\hat{\theta}_{\pi}$ for the squared loss?

  (c) Show that the Bayes estimator is consistent. What is the limiting distribution of $\sqrt{n}\left(\hat{\theta}_{\pi}-\theta\right)$ ?

  [You may use results from the course, provided you state them clearly.]

  comment

• Paper 2, Section II, J

  Principles of Statistics

  Part II, 2019

  (a) We consider the model $\{\operatorname{Poisson}(\theta): \theta \in(0, \infty)\}$ and an i.i.d. sample $X_{1}, \ldots, X_{n}$ from it. Compute the expectation and variance of $X_{1}$ and check they are equal. Find the maximum likelihood estimator $\hat{\theta}_{M L E}$ for $\theta$ and, using its form, derive the limit in distribution of $\sqrt{n}\left(\hat{\theta}_{M L E}-\theta\right)$.

  (b) In practice, Poisson-looking data show overdispersion, i.e., the sample variance is larger than the sample expectation. For $\pi \in[0,1]$ and $\lambda \in(0, \infty)$, let $p_{\pi, \lambda}: \mathbb{N}_{0} \rightarrow[0,1]$,

  $$k \mapsto p_{\pi, \lambda}(k)= \begin{cases}\pi e^{-\lambda} \frac{\lambda^{k}}{k !} & \text { for } k \geqslant 1 \\ (1-\pi)+\pi e^{-\lambda} & \text { for } k=0\end{cases}$$

  Show that this defines a distribution. Does it model overdispersion? Justify your answer.

  (c) Let $Y_{1}, \ldots, Y_{n}$ be an i.i.d. sample from $p_{\pi, \lambda}$. Assume $\lambda$ is known. Find the maximum likelihood estimator $\hat{\pi}_{M L E}$ for $\pi$.

  (d) Furthermore, assume that, for any $\pi \in[0,1], \sqrt{n}\left(\hat{\pi}_{M L E}-\pi\right)$ converges in distribution to a random variable $Z$ as $n \rightarrow \infty$. Suppose we wanted to test the null hypothesis that our data arises from the model in part (a). Before making any further computations, can we necessarily expect $Z$ to follow a normal distribution under the null hypothesis? Explain. Check your answer by computing the appropriate distribution.

  [You may use results from the course, provided you state it clearly.]

  comment

• Paper 1, Section II, J

  Principles of Statistics

  Part II, 2019

  In a regression problem, for a given $X \in \mathbb{R}^{n \times p}$ fixed, we observe $Y \in \mathbb{R}^{n}$ such that

  $$Y=X \theta_{0}+\varepsilon$$

  for an unknown $\theta_{0} \in \mathbb{R}^{p}$ and $\varepsilon$ random such that $\varepsilon \sim \mathcal{N}\left(0, \sigma^{2} I_{n}\right)$ for some known $\sigma^{2}>0$.

  (a) When $p \leqslant n$ and $X$ has rank $p$, compute the maximum likelihood estimator $\hat{\theta}_{M L E}$ for $\theta_{0}$. When $p>n$, what issue is there with the likelihood maximisation approach and how many maximisers of the likelihood are there (if any)?

  (b) For any $\lambda>0$ fixed, we consider $\hat{\theta}_{\lambda}$ minimising

  $$\|Y-X \theta\|_{2}^{2}+\lambda\|\theta\|_{2}^{2}$$

  over $\mathbb{R}^{p}$. Derive an expression for $\hat{\theta}_{\lambda}$ and show it is well defined, i.e., there is a unique minimiser for every $X, Y$ and $\lambda$.

  Assume $p \leqslant n$ and that $X$ has rank $p$. Let $\Sigma=X^{\top} X$ and note that $\Sigma=V \Lambda V^{\top}$ for some orthogonal matrix $V$ and some diagonal matrix $\Lambda$ whose diagonal entries satisfy $\Lambda_{1,1} \geqslant \Lambda_{2,2} \geqslant \ldots \geqslant \Lambda_{p, p}$. Assume that the columns of $X$ have mean zero.

  (c) Denote the columns of $U=X V$ by $u_{1}, \ldots, u_{p}$. Show that they are sample principal components, i.e., that their pairwise sample correlations are zero and that they have sample variances $n^{-1} \Lambda_{1,1}, \ldots, n^{-1} \Lambda_{p, p}$, respectively. [Hint: the sample covariance between $u_{i}$ and $u_{j}$ is $n^{-1} u_{i}^{\top} u_{j}$.]

  (d) Show that

  $$\hat{Y}_{M L E}=X \hat{\theta}_{M L E}=U \Lambda^{-1} U^{\top} Y .$$

  Conclude that prediction $\hat{Y}_{M L E}$ is the closest point to $Y$ within the subspace spanned by the normalised sample principal components of part (c).

  (e) Show that

  $$\hat{Y}_{\lambda}=X \hat{\theta}_{\lambda}=U\left(\Lambda+\lambda I_{p}\right)^{-1} U^{\top} Y$$

  Assume $\Lambda_{1,1}, \Lambda_{2,2}, \ldots, \Lambda_{q, q}>>\lambda>>\Lambda_{q+1, q+1}, \ldots, \Lambda_{p, p}$ for some $1 \leqslant q<p$. Conclude that prediction $\hat{Y}_{\lambda}$ is approximately the closest point to $Y$ within the subspace spanned by the $q$ normalised sample principal components of part (c) with the greatest variance.

  comment

• Paper 1, Section II, J

  Principles of Statistics

  Part II, 2020

  State and prove the Cramér-Rao inequality for a real-valued parameter $\theta$. [Necessary regularity conditions need not be stated.]

  In a general decision problem, define what it means for a decision rule to be minimax.

  Let $X_{1}, \ldots, X_{n}$ be i.i.d. from a $N(\theta, 1)$ distribution, where $\theta \in \Theta=[0, \infty)$. Prove carefully that $\bar{X}_{n}=\frac{1}{n} \sum_{i=1}^{n} X_{i}$ is minimax for quadratic risk on $\Theta$.

  comment

• Paper 2, Section II, J

  Principles of Statistics

  Part II, 2020

  Consider $X_{1}, \ldots, X_{n}$ from a $N\left(\mu, \sigma^{2}\right)$ distribution with parameter $\theta=\left(\mu, \sigma^{2}\right) \in$ $\Theta=\mathbb{R} \times(0, \infty)$. Derive the likelihood ratio test statistic $\Lambda_{n}\left(\Theta, \Theta_{0}\right)$ for the composite hypothesis

  $$H_{0}: \sigma^{2}=1 \text { vs. } H_{1}: \sigma^{2} \neq 1$$

  where $\Theta_{0}=\{(\mu, 1): \mu \in \mathbb{R}\}$ is the parameter space constrained by $H_{0}$.

  Prove carefully that

  $$\Lambda_{n}\left(\Theta, \Theta_{0}\right) \rightarrow^{d} \chi_{1}^{2} \quad \text { as } n \rightarrow \infty$$

  where $\chi_{1}^{2}$ is a Chi-Square distribution with one degree of freedom.

  comment

• Paper 3, Section II, J

  Principles of Statistics

  Part II, 2020

  Let $\Theta=\mathbb{R}^{p}$, let $\mu>0$ be a probability density function on $\Theta$ and suppose we are given a further auxiliary conditional probability density function $q(\cdot \mid t)>0, t \in \Theta$, on $\Theta$ from which we can generate random draws. Consider a sequence of random variables $\left\{\vartheta_{m}: m \in \mathbb{N}\right\}$ generated as follows:

  • For $m \in \mathbb{N}$ and given $\vartheta_{m}$, generate a new draw $s_{m} \sim q\left(\cdot \mid \vartheta_{m}\right)$.

  • Define

  $$\vartheta_{m+1}= \begin{cases}s_{m}, & \text { with probability } \rho\left(\vartheta_{m}, s_{m}\right) \\ \vartheta_{m}, & \text { with probability } 1-\rho\left(\vartheta_{m}, s_{m}\right)\end{cases}$$

  where $\rho(t, s)=\min \left\{\frac{\mu(s)}{\mu(t)} \frac{q(t \mid s)}{q(s \mid t)}, 1\right\}$.

  (i) Show that the Markov chain $\left(\vartheta_{m}\right)$ has invariant measure $\mu$, that is, show that for all (measurable) subsets $B \subset \Theta$ and all $m \in \mathbb{N}$ we have

  $$\int_{\Theta} \operatorname{Pr}\left(\vartheta_{m+1} \in B \mid \vartheta_{m}=t\right) \mu(t) d t=\int_{B} \mu(\theta) d \theta$$

  (ii) Now suppose that $\mu$ is the posterior probability density function arising in a statistical model $\{f(\cdot, \theta): \theta \in \Theta\}$ with observations $x$ and a $N\left(0, I_{p}\right)$ prior distribution on $\theta$. Derive a family $\{q(\cdot \mid t): t \in \Theta\}$ such that in the above algorithm the acceptance probability $\rho(t, s)$ is a function of the likelihood ratio $f(x, s) / f(x, t)$, and for which the probability density function $q(\cdot \mid t)$ has covariance matrix $2 \delta I_{p}$ for all $t \in \Theta$.

  comment

• Paper 4 , Section II, J

  Principles of Statistics

  Part II, 2020

  Consider $X_{1}, \ldots, X_{n}$ drawn from a statistical model $\{f(\cdot, \theta): \theta \in \Theta\}, \Theta=\mathbb{R}^{p}$, with non-singular Fisher information matrix $I(\theta)$. For $\theta_{0} \in \Theta, h \in \mathbb{R}^{p}$, define likelihood ratios

  $$Z_{n}(h)=\log \frac{\prod_{i=1}^{n} f\left(X_{i}, \theta_{0}+h / \sqrt{n}\right)}{\prod_{i=1}^{n} f\left(X_{i}, \theta_{0}\right)}, \quad X_{i} \sim^{i . i . d .} f\left(\cdot, \theta_{0}\right)$$

  Next consider the probability density functions $\left(p_{h}: h \in \mathbb{R}^{p}\right)$ of normal distributions $N\left(h, I\left(\theta_{0}\right)^{-1}\right)$ with corresponding likelihood ratios given by

  $$Z(h)=\log \frac{p_{h}(X)}{p_{0}(X)}, \quad X \sim p_{0} .$$

  Show that for every fixed $h \in \mathbb{R}^{p}$, the random variables $Z_{n}(h)$ converge in distribution as $n \rightarrow \infty$ to $Z(h) .$

  [You may assume suitable regularity conditions of the model $\{f(\cdot, \theta): \theta \in \Theta\}$ without specification, and results on uniform laws of large numbers from lectures can be used without proof.]

  comment

• Paper 1, Section II, J

  Principles of Statistics

  Part II, 2021

  Let $X_{1}, \ldots, X_{n}$ be random variables with joint probability density function in a statistical model $\left\{f_{\theta}: \theta \in \mathbb{R}\right\}$.

  (a) Define the Fisher information $I_{n}(\theta)$. What do we mean when we say that the Fisher information tensorises?

  (b) Derive the relationship between the Fisher information and the derivative of the score function in a regular model.

  (c) Consider the model defined by $X_{1}=\theta+\varepsilon_{1}$ and

  $$X_{i}=\theta(1-\sqrt{\gamma})+\sqrt{\gamma} X_{i-1}+\sqrt{1-\gamma} \varepsilon_{i} \quad \text { for } i=2, \ldots, n$$

  where $\varepsilon_{1}, \ldots, \varepsilon_{n}$ are i.i.d. $N(0,1)$ random variables, and $\gamma \in[0,1)$ is a known constant. Compute the Fisher information $I_{n}(\theta)$. For which values of $\gamma$ does the Fisher information tensorise? State a lower bound on the variance of an unbiased estimator $\hat{\theta}$ in this model.

  comment

• Paper 2, Section II, J

  Principles of Statistics

  Part II, 2021

  Let $X_{1}, \ldots, X_{n}$ be i.i.d. random observations taking values in $[0,1]$ with a continuous distribution function $F$. Let $\hat{F}_{n}(x)=n^{-1} \sum_{i=1}^{n} \mathbf{1}_{\left\{X_{i} \leqslant x\right\}}$ for each $x \in[0,1]$.

  (a) State the Kolmogorov-Smirnov theorem. Explain how this theorem may be used in a goodness-of-fit test for the null hypothesis $H_{0}: F=F_{0}$, with $F_{0}$ continuous.

  (b) Suppose you do not have access to the quantiles of the sampling distribution of the Kolmogorov-Smirnov test statistic. However, you are given i.i.d. samples $Z_{1}, \ldots, Z_{n m}$ with distribution function $F_{0}$. Describe a test of $H_{0}: F=F_{0}$ with size exactly $1 /(m+1)$.

  (c) Now suppose that $X_{1}, \ldots, X_{n}$ are i.i.d. taking values in $[0, \infty)$ with probability density function $f$, with $\sup _{x \geqslant 0}\left(|f(x)|+\left|f^{\prime}(x)\right|\right)<1$. Define the density estimator

  $$\left.\hat{f}_{n}(x)=n^{-2 / 3} \sum_{i=1}^{n} \mathbf{1}_{\left\{X_{i}\right.}-\frac{1}{2 n^{1 / 3}} \leqslant x \leqslant X_{i}+\frac{1}{2 n^{1 / 3}}\right\}, \quad x \geqslant 0 .$$

  Show that for all $x \geqslant 0$ and all $n \geqslant 1$,

  $$\mathbb{E}\left[\left(\hat{f}_{n}(x)-f(x)\right)^{2}\right] \leqslant \frac{2}{n^{2 / 3}} .$$

  comment

• Paper 3, Section II, J

  Principles of Statistics

  Part II, 2021

  Let $X_{1}, \ldots, X_{n} \sim$ iid $\operatorname{Gamma}(\alpha, \beta)$ for some known $\alpha>0$ and some unknown $\beta>0$. [The gamma distribution has probability density function

  $$f(x)=\frac{\beta^{\alpha}}{\Gamma(\alpha)} x^{\alpha-1} e^{-\beta x}, \quad x>0$$

  and its mean and variance are $\alpha / \beta$ and $\alpha / \beta^{2}$, respectively.]

  (a) Find the maximum likelihood estimator $\hat{\beta}$ for $\beta$ and derive the distributional limit of $\sqrt{n}(\hat{\beta}-\beta)$. [You may not use the asymptotic normality of the maximum likelihood estimator proved in the course.]

  (b) Construct an asymptotic $(1-\gamma)$-level confidence interval for $\beta$ and show that it has the correct (asymptotic) coverage.

  (c) Write down all the steps needed to construct a candidate to an asymptotic $(1-\gamma)$-level confidence interval for $\beta$ using the nonparametric bootstrap.

  comment

• Paper 4, Section II, J

  Principles of Statistics

  Part II, 2021

  Suppose that $X \mid \theta \sim \operatorname{Poisson}(\theta), \theta>0$, and suppose the prior $\pi$ on $\theta$ is a gamma distribution with parameters $\alpha>0$ and $\beta>0$. [Recall that $\pi$ has probability density function

  $$f(z)=\frac{\beta^{\alpha}}{\Gamma(\alpha)} z^{\alpha-1} e^{-\beta z}, \quad z>0$$

  and that its mean and variance are $\alpha / \beta$ and $\alpha / \beta^{2}$, respectively. ]

  (a) Find the $\pi$-Bayes estimator for $\theta$ for the quadratic loss, and derive its quadratic risk function.

  (b) Suppose we wish to estimate $\mu=e^{-\theta}=\mathbb{P}_{\theta}(X=0)$. Find the $\pi$-Bayes estimator for $\mu$ for the quadratic loss, and derive its quadratic risk function. [Hint: The moment generating function of a Poisson $(\theta)$ distribution is $M(t)=\exp \left(\theta\left(e^{t}-1\right)\right)$ for $t \in \mathbb{R}$, and that of a Gamma $(\alpha, \beta)$ distribution is $M(t)=(1-t / \beta)^{-\alpha}$ for $t<\beta$.]

  (c) State a sufficient condition for an admissible estimator to be minimax, and give a proof of this fact.

  (d) For each of the estimators in parts (a) and (b), is it possible to deduce using the condition in (c) that the estimator is minimax for some value of $\alpha$ and $\beta$ ? Justify your answer.

  comment